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(54) NETWORK CONTROL SYSTEM, CONTROLLER, AND DEVICE 

(57) When a plurality of display parts, or icons, such 
as "play" or "stop" are displayed on a screen, informa- 
tion about the positional relationship in which these 
icons will be laid out on the screen is first assigned as 
an attribute of each icon. Also, using a certain icon as a 
reference, information about the layout of the icons, 
such as whether they are above, below, to the left, or to 
the right of this icon, is assigned as an attribute of the 
icon serving as the reference. In this way it is possible to 
vary the layout of a display according to the size of the 
display screen. 
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Descripti n 
TECHNICAL FIELD 

[0001] This invention relates to a network control 5 
system in which units* 1 1 connected on a network are 
operated through the network, and more particularly to 
a unit control system that makes use of a graphical user 
interface (GUI) that supports unit operation by the user 
by means of graphics, characters, or the like on a 
screen. 

BACKGROUND ART 

[0002] Recent years have seen the debut of unit 
control systems in which graphics (icons) consisting of 
characters or the like, or screen display information 
showing the function of the units is displayed on a tele- 
vision screen, and a unit is controlled by using a televi- 
sion remote control to select from among these 
graphics. Another network system that has debuted 
makes use of IEEE 1394-1995 to connect DVC or other 
such digital units and transmit and receive audio/video 
data. 

[0003] A conventional network control system is 
disclosed in Japanese Laid-Open Patent Application 9- 
149325. 

[0004] An example of a conventional network con- 
trol system will be described below. 
[0005] With AV units, such as with a digital interface 
of IEEE 1394 standard or the like, the various AV units 
are connected by a serial bus periodically given equal 
communication opportunities through a bidirectional 
packet communication system with other AV units with- 
out the various AV units being switch-connected. 
[0006] Here, the various AV units each store their 
own screen display data, this screen display data is 
transmitted to a controller having a graphic display func- 
tion (such as a television receiver) at the request of the 
controller, and the controller displays this screen display 
data. 

[0007] The controller also has a function for inquir- 
ing about the data needed for the display of the con- 
nected AV units, and a function for controlling the 
display screen on the basis of the screen display data 
from the AV units. 

[0008] The AV units each have a recording medium 
for storing screen display data, and a function for select- 
ing the suitable screen display data with respect to the 
inquiry about screen display data from the controller. 
[0009] With a conventional network control system 
configured this way, the screen display data is first 
stored in each device (AV unit), and is then outputted 
according to the display requests from the controller, 
(television receiver), which results in the graphics of 
each individual device (AV unit) being displayed on the 
screen of the controller. 



DISCLOSURE OF THE INVENTION 

Technical Problems Which the Invention is Intended to 
Solv 

[0010] A problem with the above structure, how- 
ever, was that when the screen size of a controller was 
smaller than the screen size that a device was sup- 
posed to have, the information of the device could not 
be displayed properly. Another problem was that screen 
display data had to be incorporated from the device dur- 
ing the display of an operation screen for operation by 
the user, which resulted in poor response. 
[0011] Furthermore, the method for controlling a 
device was not normally disclosed, and while a control- 
ler could display the information of a device with the 
above structure, there was the problem that it was diffi- 
cult for the controller to control the device unless the 
method for controlling the device was disclosed. A par- 
ticular problem was that when video or audio stream 
data was sent out from a device, because the method 
for determining which region of the transmission line it 
was being sent to was not disclosed, it was difficult for 
the controller or another display device to receive the 
stream data sent by the device. 

[0012] The present invention was conceived in light 
of the above problems, and an object thereof is to pro- 
vide a network control system with a simple structure, 
which can respond quickly to operation by a user and 
with which an operating environment that is easy for a 
user to understand can be constructed, even when the 
screen size that a device is supposed to have is different 
from the screen size of the controller. 
[001 3] It is also an object of the present invention to 
provide a network control system with a simple struc- 
ture, with which an operating environment that is suited 
to various functions or various units can be constructed, 
which can control various functions or various units with 
ease, and with which a controller and a device can 
share channel information for a transmission line by a 
simple method, and to provide a device and a controller 
that constitute this network system. 

Method for Solution Thereof 

[0014] In order to solve the above-mentioned prob- 
lems, the network control system of the present inven- 
tion is characterized in that the devices have a plurality 
of display parts that constitute an operation screen for 
the devices, the display parts have layout information 
showing the layout relationship to be assumed with the 
other display parts, the controller reads the display parts 
from the devices, and the display parts are displayed on 
a display screen according to this layout information. 
[0015] It is also characterized in that the layout 
information is relationship information having the same 
information for a plurality of display parts to be laid out 
in proximity. 
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[0016] It is also characterized in that the layout 
information compris s identifiers of the display parts to 
be laid out in proximity. 

[0017] It is further characterized in that the display 
parts have identifiers for the display parts to be laid out 
in proximity in the directions of up, down, left, and right. 
[0018] It is also characterized in that one display 
part has a plurality of sets of layout information. 
[0019] It is also characterized in that the devices 
have a plurality of display parts that constitute an oper- 
ation screen for the devices, a set consisting of a 
number of display parts is shown from among the dis- 
play parts, there are configuration set parts having infor- 
mation showing the priority order of display, the 
controller reads the display parts and the set informa- 
tion from the devices, and the display parts are dis- 
played on a display screen according to information 
about the configuration set parts. 
[0020] It is further characterized in that the configu- 
ration set parts have a hierarchical structure. 
[0021] It is also characterized in that the devices 
have a menu consisting of a plurality of display parts 
showing an operation screen for the devices, the con- 
troller reads the display parts from the devices, and 
when the menu is larger than the display screen of the 
controller, the display of the controller is divided into a 
plurality of pages the same size as or smaller than the 
display screen, and the controller produces information 
about navigation between the plurality of pages and dis- 
plays this information on the display screen. 
[0022] It is also characterized in that the devices 
have display parts having manipulation information for 
changing the display of the operation screen through 
operation by the user, and a menu consisting of a plural- 
ity of the display parts showing the operation screen, 
the controller reads the display parts from the devices, 
the menu of the devices is displayed on the display 
screen, and the display of the display screen is changed 
through the operation by the user according to the 
manipulation information. 

[0023] It is also characterized in that the manipula- 
tion information comprises identifiers showing the other 
display parts. 

[0024] It is also characterized in that the devices 
have a plurality of menus, and the menus are equipped 
with display parts having as manipulation information 
identifiers that show the other menus. 
[0025] It is also characterized in that the devices 
have help menus that explain the functions exhibited by 
the various display parts, and the display parts have as 
manipulation information identifiers that show the help 
menus. 

[0026] In order to solve the above-mentioned prob- 
lems, the network control system of the present inven- 
tion is characterized in that the devices have display 
parts that contain channel information indicating the 
channel of the transmission line and that constitute an 
operation screen for the devices, the controller reads 



the display parts from the devices, and the channel of 
the transmission lin is set according to the channel 
information in the handling of the data shown by the dis- 
play parts. 

5 [0027] It is also characterized in that the devices 
have a plurality of display parts that constitute an oper- 
ation screen for the devices and configuration set parts 
that have channel information indicating the channel of 
the transmission line and that show a set consisting of a 

w number of display parts from among the display parts, 
the controller reads the configuration set parts and the 
display parts from the devices, and the channel of the 
transmission line is set according to the channel infor- 
mation in the handling of the data shown by the display 

15 parts belonging to the configuration set parts. 

[0028] It is also characterized in that the devices 
have a plurality of display parts that show an operation 
screen for the devices and menus listing channel infor- 
mation indicating the channel of the transmission line, 

20 the controller reads the display parts and the menus 
from the devices, and the channel of the transmission 
line is set according to the channel information in the 
handling of the data shown by the display parts belong- 
ing to the menus. 

25 [0029] It is also characterized in that the display 
parts, configuration set parts, or menus have flags 
showing the type of stream. 

[0030] It is also characterized in that the devices 
have content display parts that constitute an operation 

30 screen for the devices, show the content handled by the 
devices, and list control codes specifying the content, 
the controller reads the display parts from the devices, 
and the controller uses the control codes to direct the 
transmission and receipt of the content to the devices in 

35 the operation of the content display parts by the user on 
the display screen. 

[0031] It is also characterized in that the content 
display parts have channel information indicating the 
channel of the transmission line over which the content 
40 shown by the content display parts are transmitted and 
received. 

[0032] It is also characterized in that the devices 
have display parts that constitute an operation screen 
for the devices, show the content handled by the 

45 devices, and list control codes specifying the content, 
and content configuration set parts that show a set con- 
sisting of a plurality of the content display parts, the con- 
troller reads the display parts and the content 
configuration set parts from the devices, the content dis- 

50 play parts are displayed on a display screen according 
to the information shown by the configuration set parts, 
and the controller uses the control codes to direct the 
transmission and receipt of the content to the devices in 
the operation of the display parts by the user on the dis- 

55 play screen. 

[0033] It is also characterized in that the devices 
have display parts that constitute an operation screen 
for the devices, show the content handled by the 
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devices, and list control codes specifying the content, 
and content menus that show the functions of the 
devices and codes that indicate one or a plurality of the 
display parts, the controller reads the content menus 
and the display parts from the devices, the content 
menus are displayed on a display screen, and the con- 
troller uses the control codes to direct the transmission 
and receipt of the content to the devices in the operation 
of the display parts by the user on the display screen. 
[0034] / It is also characterized in that the devices 
have display parts that constitute an operation screen 
for the devices and content menus that show the func- 
tions of the devices and codes that indicate one or a plu- 
rality of the display parts, the controller requests the 
right to use the devices, and the devices transmit a 
menu list containing identifiers for the display parts that 
constitute the menus of the devices to the controller 
when the devices grant the usage right. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0035] 

Figure 1 is a structural diagram of a function infor- 
mation table in the first and sixth embodiments; 
Figure 2 is a block diagram of a device in the net- 
work control system in the first embodiment; 
Figure 3 is a block diagram of a controller in the net- 
work control system in the first embodiment; 
Figure 4 is a system configuration diagram illustrat- 
ing an example of the network control system in the 
first embodiment; 

Figure 5 is an explanatory diagram of the network 
control system in the first embodiment; 
Figure 6 is a structural diagram of display parts in 
the second embodiment; 

Figure 7A is an explanatory diagram illustrating the 
display parts and relationship information in the 
second embodiment; 

Figure 7B is an explanatory diagram of a normal 
screen display in the second embodiment; 
Figure 7C is a first explanatory diagram of a small 
screen display in the second embodiment; 
Figure 7D is a second explanatory diagram of a 
small screen display in the second embodiment; 
Figure 8 is a structural diagram of display parts in 
the third embodiment; 

Figure 9 is a structural diagram illustrating the func- 
tion menu and configuration set parts in the fourth 
embodiment; 

Figure 10A is an explanatory diagram of a normal 
screen display in the fourth and seventh embodi- 
ments; 

Figure 1 0B is a first explanatory diagram of a small 
screen display in the fourth and seventh embodi- 
ments; 

Figure 10C is a second explanatory diagram of a 
small screen display in the fourth and seventh 



embodiments; 

Figure 11A is a structural diagram of one display 
part having operation information in the fifth embod- 
iments; 

5 Figure 11 B is an explanatory diagram of the display 

screen of the main menu in the fifth embodiment; 
Figure 1 1C is an explanatory diagram of the display 
screen of menu 1 in the fifth embodiment; 
Figure 12 is a structural diagram of a display part in 

10 the fifth embodiment; 

Figure 13A is an explanatory diagram illustrating 
display parts and manipulation information in the 
fifth embodiment; 

Figure 13B is an explanatory diagram of the screen 
15 display in the initial state in the fifth embodiment; 

Figure 13C is an explanatory diagram of the screen 
display during right key selection in the fifth embod- 
iment; 

Figure 14 is a structural diagram of a function infor- 
20 mation table in the sixth embodiment; 

Figure 15 is an explanatory diagram explaining con- 
trol related to the function menu in the sixth embod- 
iment; 

Figure 16 is an explanatory diagram of the network 
25 control system in the sixth embodiment; 

Figure 17 is a structural diagram of the function 
menu and configuration set parts in the seventh 
embodiment; 

Figure 18 is an explanatory diagram of explaining 
30 control related to the function menu in the seventh 
embodiment; 

Figure 19 is a structural diagram of the function 
menu and configuration set parts in the seventh 
embodiment; 

35 Figure 20 is a structural diagram illustrating the 
function menu and configuration set parts in the 
eighth embodiment; 

Figure 21 is an explanatory diagram of a normal 
screen display in the eighth embodiment; 
40 Figure 22 is a structural diagram of the configura- 
tion set parts and display parts in the eighth embod- 
iment; and 

Figure 23 is a structural diagram of the function 
menu and display parts in the eighth embodiment. 

45 

BEST MODE FOR CARRYING OUT THE INVENTION 

[0036] The network control system in an embodi- 
ment of the present invention will now be described 
so through reference to the figures. 

First Embodiment: General layout information 

[0037] The first embodiment will be described using 
55 Figures 1 to 5. When a plurality of display parts, or 
icons, such as "play" or "stop** are displayed on a 
screen, information about the positional relationship in 
which these icons will be laid out on the screen is first 
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assigned as an attribute of each icon. In the first embod- 
iment, an overall description of a network control sys- 
tem will be given using relationship information as a 
typical example of layout information. 
[0038] Figure 1 illustrates the structure of a function 
information table in this embodiment, Figure 2 is a block 
diagram of a device in the network control system in this 
embodiment, and Figure 3 is a block diagram of a con- 
troller in the network control system in this embodiment. 
The structure and operation of the network system in 
this embodiment will now be described using Figures 1 , 
2, and 3. 

[0039] The term "device" used in this Specification 
refers to something that is being controlled, while "con- 
troller" refers to what is controlling these. A device and 
a controller may both be present in a single unit, or there 
may be just one or the other. A "unit" corresponds to a 
single node on a transmission line, and an apparatus 
may be constituted such that there are a plurality of 
nodes within a single case. 

[0040] First, in Figure 2, 1 is a transmission line, 2 is 
a packet transmitting and receiving means, 3 is a syn- 
chronous data transmitting and receiving means, 4 is a 
device signal processing means, 5 is an asynchronous 
data transmitting and receiving means, 6 is a device 
asynchronous data processing means, 7 is unit configu- 
ration information, 8 is function information tables, 9 is a 
unit internal control means, 15 is a rewritable memory 
region (ROM) in which some of the function information 
tables 8 are laid out, 16 is a rewritable memory region 
(RAM) in which some of the function information tables 
8 are laid out, and 17 is a function information manage- 
ment means. 

[0041] Here, transmission line 1 is a serial bus 
(1394 bus) as set forth, for example, in the IEEE 1394 
standard (IEEE 1394-1995 and superseding standards 
compatible therewith), and is able to transmit and 
receive both synchronous data and asynchronous data 
by time sharing or another such method. Furthermore, 
synchronous data can be transmitted using a plurality of 
channels divided by time sharing or another such 
method, and the bands of these various channels can 
be set individually. The transmission line 1 does not 
necessarily have to be a 1394 bus, and the transmission 
lines used in ATMs, the Ethernet, infrared transmission, 
or the like may be used instead. 
[0042] The packet transmitting and receiving 
means 2 provides a physical and electrical interface 
with the transmission line 1, and also mediates the bus 
usage right, performs cycle control for synchronous 
transmission, and so on. Furthermore, the packet trans- 
mitting and receiving means 2 picks out and receives 
packets from the transmission line 1 according to the 
address, and transmits packets over the transmission 
line 1. 

[0043] The synchronous data transmitting and 
receiving means 3 manages the transfer rate (division of 
data) or adds headers during transmission. For 



instance, when a 1394 bus with an AV protocol (IEC 
61883) standard is used, CIP (Common Isochronous 
Packet) headers are added by the synchronous data 
transmitting and r ceiving means 3. Conversely, wh n 

5 data is being received, th synchronous data transmit- 
ting and receiving means 3 sorts the received packets in 
the proper order, removes headers, and so on. 
[0044] The device signal processing means 4 
receives synchronous data from the synchronous data 

w transmitting and receiving means 3, and performs sig- 
nal processing according to the device. For instance, if 
this device is a recording and playback unit such as a 
digital VTR, then it records synchronous data to a 
recording medium (such as a magnetic tape). The 

15 device signal processing means 4 also fetches synchro- 
nous data from a recording medium, broadcast waves, 
or the like, and sends it to the synchronous data trans- 
mitting and receiving means 3. 

[0045] The asynchronous data transmitting and 

20 receiving means 5 performs transaction processing of 
asynchronous data according to the protocol of the 
transmission line 1. For instance, with a 1394 bus, it 
processes read transactions, write transactions, lock 
transactions, and so forth. Here, the asynchronous data 

25 transmitting and receiving means 5 may consist of soft- 
ware. The device asynchronous data processing means 
6 processes the asynchronous data received from the 
asynchronous data transmitting and receiving means 5, 
and transmits it to the suitable constituent elements of 

30 this device. For example, if the asynchronous data 
received by the device asynchronous data processing 
means 6 comprises control codes and operation infor- 
mation from the user, the validity thereof is evaluated, 
and if these are valid, the device asynchronous data 

35 processing means 6 outputs a directive to the unit inter- 
nal control means 9 to execute a function corresponding 
to these control codes and operation information from 
the user. 

[0046] Here, the control codes and operation infor- 
40 mation from the user may be transmitted to the function 
information management means 17, and at this point 
the function information management means 17 evalu- 
ates these for validity, and if it finds them valid, it directs 
the unit internal control means 9 to execute the func- 
45 tions indicated by these. 

[0047] The device asynchronous data processing 
means 6 also reserves and sets the channels or bands 
used for synchronous data. 

[0048] Next, when there is a request for unit config- 
50 uration information 7 from the controller, the device 
asynchronous data processing means 6 sends the 
information contained in the unit configuration informa- 
tion 7 to the controller via the asynchronous data trans- 
mitting and receiving means 5 or the like according to 
55 the request received via the asynchronous data trans- 
mitting and receiving means 5 or the like. 
[0049] Furthermore, the d vice asynchronous data 
processing means 6 sends asynchronous data from the 
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constituent elements within the devic to the asynchro- 
nous data transmitting and receiving means 5 on the 
basis of a directive from the unit internal control means 
9. 

[0050] Here, the asynchronous data transmitting 
and receiving means 5 and the device asynchronous 
data processing means 6 may consist of a single 
means. 

[0051] The unit configuration information 7 indi- 
cates the configuration information for a unit, and is 
described, for instance, according to the rules given for 
the configuration ROM of CSR (Command and Status 
Registers) architecture in the ISO/IEC 13213:1994 
standards. When a 1394 bus is used, there is a unit 
directory containing information about the bus to which 
this unit corresponds, that is, whether a bus manager or 
isochronous operation is supported, and information 
about whether an AV protocol is supported, and there is 
a unique ID which is an identifier for this unit. The unit 
configuration information 7 also contains the leading 
address of the ROM 1 5 in order to show where the func- 
tion information tables 8 are. The function information 
tables 8 are tables of information for configuring the 
operation screen of this device. 

[0052] These function information tables 8 include 
objects necessary for the operation of the device, iden- 
tifiers (ID) for identifying these objects, and so on. 
[0053] Here, each object has a hierarchical struc- 
ture in list format, and in this Specification, the various 
display parts and the intonation and lists themselves will 
be collectively referred to as objects. 
[0054] The term "display part" here refers to some- 
thing that is locally displayed on a screen, such as an 
icon, button, slider, checkbox, text entry, or the like, and 
there is still picture data such as the operating button of 
a unit, text data indicating a unit or the like, audio data 
such as sound effects, and program codes including still 
picture data or text data. The information in the function 
information tables 8 is transferred to the controller via 
the 6, the asynchronous data transmitting and receiving 
means 5, or the like according to a request from the 
controller on the transmission line 1 . 
[0055] These function information tables 8 are laid 
out in the ROM 15 and the RAM 16. In the ROM 15 is 
stored information that is specific to the device and does 
not need to be rewritten frequently, that is, objects such 
as still picture data showing the operating button for a 
unit. Furthermore, this ROM 15 may comprise a flash 
ROM, in which case the function of the unit itself can be 
rewritten. 

[0056] Also, the unit internal control means 9 or the 
controller on the transmission line 1 writes objects as 
needed via the function information management 
means 17 to the RAM 16 in which this function informa- 
tion tables 8 are laid out. The information written here is 
content information or status information. 
[0057] In the case of an STB. for example, this con- 
tent information is information about programs currently 



being broadcast (information such as the program title, 
title screen, theme music, summary, cast cr dits, or the 
like), and in the case of DVD, this is information about 
the content recorded on th DVD disk (information such 
5 as the program title, title picture, them music, sum- 
mary, cast credits, or the like). 

[0058] In the case of a VTR, for example, this status 
information is an object such as a display part that 
shows the status of the unit (play, rewind, timer record- 

10 ing). Furthermore, any information necessary for net- 
work control, such as identifier information about the 
controller using this device, or the date, time, channel 
number, and other such timer recording information, 
may also be written here. The function information man- 

15 agement means 1 7 converts the address of the ROM 1 5 
or RAM 16 and the identifier (ID) of an object. Further, 
the address is not merely converted, and when, for 
example, a certain display part is rewritten and the data 
size is so large that it cannot be written to the original 

20 address region, a new address is assigned. 

[0059] Thus, each object can be read or written with 
the ID of the object from the device asynchronous data 
processing means 6, the unit internal control means 9, 
or the controller on the transmission line 1 . 

25 [0060] If the address of each object or the like is 
known, then the address of the ROM 1 5 or the RAM 1 6 
may be used to read or write it. Further, these may be 
combined to read or write a display part or the like, or 
reading and writing may be accomplished by a relative 

30 address within the display part indicated by the ID. 

[0061] Also, when the function information manage- 
ment means 17 performs the management of an object 
ID, such as when a new object is added, this object is 
given an ID that is not shared by anything else, and con- 

35 versely, when an object is deleted, the ID of this object 
is rendered invalid. 

[0062] Also, the function information management 
means 17 can be configured such that when a display 
part is changed, the changed display part information 

40 (the ID of the object, or the ID and the object itself) is 
sent to the controller, in which case there is no need for 
the controller to constantly monitor an object that might 
change, reducing the processing of the controller, and 
allowing the system to easily accommodate an object 

45 that shows content information or status information 
which changes often. 

[0063] The unit internal control means 9 serves to 
control the various constituent elements including the 
mechanisms and so forth within this device, and if the 

so data received by the device asynchronous data 
processing means 6 is a control code indicating the 
operation of the function of the device, then the opera- 
tion is carried out according to this control code at the 
directive of the device asynchronous data processing 

55 means 6. 

[0064] The operation of a device with respect to a 
request from the controller or the like is as follows. First, 
when the device is connected to the transmission line 1 , 
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or when the controller is connected to the transmission 
line 1, the controller first reads the unit configuration 
information 7 of th device, confirms th location of the 
function information table 8, and reads the content of 
the function information table 8. 

[0065] The structure may be such that the unit con- 
figuration information 7 does not contain the address 
information of the function information table 8, and only 
the existence of the function information table 8 is indi- 
cated, in which case the controller issues a command to 
the device requesting the function information table 8, 
and acquires the information in the function information 
table 8. in this case, the structure may be such that a 
command is issued requesting only a part of the func- 
tion information table 8, such as a display part, and only 
the display part and its ID are acquired. 
[0066] When a control code and user operation 
information are received from the controller, the 
processing indicated by this control code and the user 
operation information is carried out according to the sta- 
tus. 

[0067] Here, for an object that is a display part 
showing the function of a device, when the control code 
of this object is sent from the controller along with the 
operation "select" of the user, the asynchronous data 
transmitting and receiving means 5 outputs a directive 
to the unit internal control means 9 so that the function 
shown by this object will be executed. 
[0068] Thus, a device need only present the func- 
tion information table 8 with respect to a request from 
the controller for GUI (Graphical User Interface) infor- 
mation, so the load on the device can be reduced. For 
the various functions of a device, there is no need to 
specify a command with a standardized group or the 
like, and even with a device having new functions that 
cannot be foreseen today, it is possible to use these new 
functions easily through the transmission line 1. 
[0069] The constituent elements of the synchro- 
nous data transmitting and receiving means 3, the 
device signal processing means 4, and so on here may 
be configured as desired, or eliminated, according to 
the function of the device. Also, the various means may 
be configured either as hardware or software. 
[0070] The function of the device was specified 
here by the user operation information or the control 
code of the object, but it is also possible for the structure 
to be such that "select" is the only operation by the user 
that is recognized, in which case the function of the 
device can be specified with just the control code of the 
device, so the function of that device can be executed 
with just this control code, and the size of the packet that 
is transmitted can be smaller. 

[0071] Also, a control code was used here as the ID 
of the object, but this control code may be set for the 
device as desired, and for example may comprise a 
number assigned according to the type of function of the 
device, and a serial number for each type, or a unique 
control code used internally in the device may be used, 



in which case the various functions will be asier to 
mount in the device. 

[0072] Figure 3 is a block diagram of a controller in 
the network control system in this embodiment. Her , 

5 10 is a controller signal processing means, 1 1 is a con- 
troller asynchronous data processing means, 12 is a 
function information table management means, 13 is a 
function data base 13, and 14 is a display/function 
selection means. In Figure 3, those constituent ele- 

w ments that are the same as in Figure 2 are numbered 
the same, and will not be described again. 
[0073] The controller signal processing means 10 
receives synchronous data from the synchronous data 
transmitting and receiving means 3, and performs sig- 

15 nal processing according to this controller. For instance, 
if this controller is a video display unit such as a video 
monitor, the synchronous data (such as an MPEG2 
stream) is decoded and displayed on the screen. 
[0074] The controller asynchronous data process- 

20 ing means 11 processes the asynchronous data 
received from the asynchronous data transmitting and 
receiving means 5, and transmits it to the suitable con- 
stituent elements of this controller. The controller asyn- 
chronous data processing means 1 1 also reserves and 

25 sets the channels or bands used for synchronous data. 
[0075] The controller asynchronous data process- 
ing means 1 1 also receives the contents of the function 
information table 8 for the devices, information about the 
devices on the transmission line 1 , such as the connec- 

30 tion of a new device or the removal of an existing device, 
and so on from the packet transmitting and receiving 
means 2 via the asynchronous data transmitting and 
receiving means 5, and transmits it to the function infor- 
mation table management means 12. 

35 [0076] Further, the controller asynchronous data 
processing means 11 sends the asynchronous data 
from the constituent elements in the controller to the 
asynchronous data transmitting and receiving means 5 
on the basis of a directive from the display/function 

40 selection means 14. Here, the asynchronous data 
transmitting and receiving means 5 and the controller 
asynchronous data processing means 11 may comprise 
a single means. 

[0077] When the controller and the device are 
45 located within the same unit, either the controller knows 
the function of the device in that unit, or the internal con- 
trol of the unit is carried out directly by the unit internal 
control means 9, so there is a function information table 
8 for the device, but it is not registered in the function 
so data base 13 inside this unit. The function information 
table 8 inside this controller should have the information 
of the function information table 8 of the device, but 
these sets of information do not necessarily have to be 
the same. 

55 [0078] In this case, the location of the function infor- 
mation table 8 within the device may be noted in the unit 
configuration information 7, and registered ahead of 
time in the function data base 13. 
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[0079] When the controller and the device are 
located within th same unit, the controller signal 
processing means 10 and the device signal processing 
means 4, and the controller asynchronous data 
processing means 11 and the device asynchronous 5 
data processing means 6 may comprise a single unit. 
[0080] The function information table management 
means 12 manages the function information table 8 
received from a device on the transmission line 1 , and 
when information is received from the controller asyn- 10 
chronous data processing means 1 1 to the effect that a 
new device has been connected, a directive is outputted 
to the controller asynchronous data processing means 
11 to read the function information table 8 of this new 
device. is 
[0081] Then, when the function information table 8 
of a new device has been read, this function information 
table 8 is registered in the function data base 13. Also, 
when information is received to the effect that an exist- 
ing device on the transmission line 1 has been removed, 20 
for example, the corresponding function information 
table 8 is deleted from the function data base 1 3. 
[0082] The structure here may be such that when 
an existing device is removed, the function information 
table 8 is not deleted but is instead stored in a memory 25 
means within the controller, or the structure may be 
such that when this device is connected again, this 
device is recognized by a device identifier or the like, 
and the function information table 8 is read from the 
memory means within the controller and registered in 30 
the function data base 13, both of which allow quick reg- 
istration of the connected unit. 

[0083] The function data base 13 is disposed in a 
rewritable memory space, and the function information 
table 8 received from a device is configured as a data 35 
base. By using the function information table manage- 
ment means 12 to search this data base, it is possible to 
fetch objects such as information about various devices ' 
or about various units, ID corresponding to these 
objects, display parts for notifying the user of these 40 
objects, display parts that are to be displayed when the 
user operates these display parts, control codes to be 
transmitted, and so on. 

[0084] The function data base 13 does not neces- 
sarily have to contain all of the information of the func- 45 
tion information table 8 all the time, and may contain 
only the required portion. 

[0085] The display/function selection means 14 
notifies the user of display parts(video, audio, character 
information, etc.) showing GUI information about a func- so 
tion or GUI information about a device on the controller 
screen, and also selects devices and functions accord- 
ing to operation by the user, directs the execution of var- 
ious functions, and so forth. 

[0086] It is also possible to display/play data 55 
received from the controller signal processing means 10 
(such as video or audio data) or data received from the 
controller asynchronous data processing means 1 1 . 



[0087] In this case, the GUI information and the like 
may be displayed in overlay with the video data receiv d 
from the controller signal processing means 10, or the 
FUI information display scr en and the video data dis- 
play screen may be alternat ly displayed through 
switching by directive from the user or the like. 
[0088] Furthermore, a directive is sent to the func- 
tion information table management means 12 to search 
the function information table 8 within the controller, and 
display on the screen the display parts showing the 
devices on the transmission line 1 or the functions of 
these devices (device titles, function titles, display-use 
still pictures, etc.). 

[0089] When the user selects a display part show- 
ing a device, the menu of this device is read from the 
function information table 8 and displayed on the 
screen. 

[0090] When the user selects a display part show- 
ing a function, the control code corresponding to this 
display part obtained from the function information table 
8 and the operation information obtained from the user 
are issued through the controller asynchronous data 
processing means 1 1 and so on. The display/function 
selection means 14 receives the response of the device 
with respect to this control code and the user operation 
information via the controller asynchronous data 
processing means 1 1 , and if there is a directive in this 
response to change a display part from the device (such 
as notification of the ID of a display part to be changed), 
then the display part to be changed is retrieved from the 
device according to this directive, and the suitable dis- 
play part with respect to the status of the device at the 
present time, for example, is displayed on the display 
screen to notify the user. 

[0091] The controller does not necessarily have to 
under every function of a device. For instance, even with 
a device having a new function that cannot be foreseen 
at present, the controller will be able to fetch the display 
part for this new function from the function information 
table 8 and display it on the screen to notify the user. 
[0092] If the user understands a new function 
through this display part and selects this function, the 
display/ function selection means 14 of the controller 
will refer to the function information table 8, obtain a 
control code corresponding to this new function, and 
issue this control code and the operation of the user to 
the device, allowing this new function to be executed by 
the device. Thus, adopting the above structure makes it 
possible for a user to execute even new functions that 
cannot be foreseen at the present time. 
[0093] The constituent elements here, such as the 
synchronous data transmitting and receiving means 3 
and the controller signal processing means 10, may be 
configured as desired, or eliminated, according to the 
functions of the controller. 

[0094] Figure 4 illustrates the system configuration 
in the network control system in this embodiment. 
[0095] In Figure 4, 21 is a television, 22 is a remote 
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control for the television, 23 is a personal computer 
(PC), 31 is a recordable DVD, 32 is DV-format digital 
VTR (DVHS), 33 is a VHS-format digital VTR (DVHS), 
34 is a DV-format digital movie (DVC movie), and 35 is a 
set top box (STB) for CS digital broadcasts or the like. 5 
These are collectively referred to as video/audio/ infor- 
mation units. 

[0096] These video/audio/information units are con- 
nected by the transmission line 1 to form an AVC sys- 
tem. Video/audio/information units are not limited to the 10 
units listed above, and include current units in the fields 
of video, audio, and information (such as printers and 
mini-disks), as well as all units that will come out in the 
future. 

[0097] The television 21 here is a unit consisting of 15 
a controller and a device (a ground wave tuner and a 
video monitor), and the user uses the remote control 22 
to give directives to the display/function selection 
means 14. The PC 23 is a unit consisting of a controller 
and devices (a modem that provides an interface with a 20 
telephone line, a video monitor, or the like), and the user 
uses a keyboard, a mouse, or the like to give directives 
to the display/ function selection means 14. 
[0098] The television 21 and the PC 23 here are 
defined as units having integrated devices and control- 25 
lers, and of the functions of a device within a unit, those 
functions that can be used from another unit are noted 
in the function information table 8 but are not them- 
selves registered in the function data base 13 of the 
controller component within the unit. The television 21 30 
and the PC 23 may also be defined as units consisting 
of devices and a controller, with each device in a unit 
having a function information table 8, and with each . 
function information table 8 within a unit being regis- 
tered in the function data base 1 3 of the controller within 35 
the unit. 

[0099] The DVD 31 and DVC movie 34 are devices 
capable of recording and playing back AV data. The 
DVC 32 and the DVHS 33 are units consisting of 
devices capable of recording and playing back AV data 40 
and devices having a digital broadcast tuner function. 
The STB 35 is a device having a tuner function for 
receiving CS digital broadcasts. 

[01 00] The DVD 31, DVC 32, DVHS 33, DVC movie 
34, and STB 35 here are termed devices, but if they are 45 
at least capable of operations in which the user selects 
the functions of other devices with a touch panel or 
remote control, or producing an environment in which 
other devices can be operated with a liquid crystal panel 
or the like, then they may be termed a unit including a 50 
controller and a device. 

[0101] Another possible structure is one in which 
these units have a unit-use remote control and include 
processing functions as a controller, only audio and 
graphic displays are displayed on a monitor by analog 55 
connection, and the user operates the remote control of 
the unit while looking at this screen. In this case, this 
unit may include a controller and a device. Figure 1 is an 



xplanatory diagram of the function information table in 
this embodiment. 

[0102] In Figur 1 , 50 denotes th device informa- 
tion parts, 51 denotes the configuration set parts, and 
52 denotes the display parts. 

[0103] Each part here consists of an object in list 
format, and data objects with no sub-objects (such as 
text objects or still picture objects) and lists are collec- 
tively referred to as objects. 

[0104] Each object has in its header portion an 
identifier (ID) that identifies that particular object, type 
information indicating the type of object, attribute infor- 
mation indicating the configuration (such as whether 
there are any sub-objects), size information indicating 
the size of the object, and so forth. Each object also 
indicates a sub-object by means of ID information within 
an entry made in a list. 

[01 05] Objects with a small amount of data, such as 
a text object, may themselves be entered in the entry 
portion within a list. The unique information of each list 
is entered in the header within each list, for example, 
and the unique information of a data object is noted in 
an entry. Unique information may be given in an object 
or in an entry. 

[0106] The device information parts 50 are parts 
showing information about devices, and the types of 
commands and protocols supported by the devices, the 
device types in which the types of these devices are 
encoded, version information about these devices, and 
so on are noted as unique information in the device 
information lists in these device information parts 50. 
The device type may also indicate a code or character 
string given as the subunit_type of the AV/C Digital 
Interface Command Set (AV/C-CTS) discussed by the 
1394 TA (1394 Trade Association), for example. 
[0107] It is also possible for the device information 
parts 50 to contain information about the function infor- 
mation table 8 itself, and the support level or size of this 
function information table 8, the maximum transfer 
quantity sent asynchronously at one time by this unit, or 
the like may be noted as unique information. 
[0108] This information may be entered in the 
device information lists here, or a route list may be pro- 
duced as a new fist for these device information lists, 
and the information entered therein. 
[01 09] In this case, before the controller reads all of 
the function information table 8, the controller can 
decide whether a level can be supported, how much 
memory space has to be reserved, how much the one- 
time transfer quantity should be, and so forth merely by 
looking at these parts, which makes it possible to elimi- 
nate wasted transfer. 

[0110] The device information parts 50 may also be 
configured such that they can be directly read and writ- 
ten from the controller via the transmission line 1 as part 
of the unit configuration information. Even with a device 
not having a menu that shows the functions, it is possi- 
ble to contain the information of the d vice itself, allow- 
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ing uniformity to be pr s rved between units. 
[0111] Th device information parts 50 also have 
user interface information about the devices, and the 
display parts 52, such as d vice names that express the 
names of these devices in character strings, or model 
names in which the manufacturer has indicated the 
model number of a product, are laid out here, with each 
of these being a text object. 

[0112] There are also display parts 52 here such as 
still picture objects showing devices, such as icons of 
devices, as still picture objects. These text objects, still 
picture objects, and so forth have entries in the device 
information list. There may be audio objects here, and 
data such as music showing these devices may also be 
laid out. 

[0113] Simulated moving pictures consisting of a 
plurality of still pictures can also be used in place of still 
picture objects even with the function menu 51 , the dis- 
play parts 52, and so on. Each object here was said to 
have an entry in the device information list, but objects 
with the same purpose may instead define a list and 
have entries in this list. Furthermore, there may be vari- 
ous lists according to the GUI classification (display, 
selection, in operation, in use, error, and so on), in 
which case a display that is easier for the user to under- 
stand can be provided. 

[0114] Next, the function menu 51 shows a menu 
that is a set of display parts 52 showing the functions of 
a device, and comprises a function menu list. The func- 
tion menu list is linked from an entry in the device infor- 
mation list, and has entries of the display parts 52 for 
showing this list itself. Thus, the function menu can be 
accessed from the device information parts 50. 
[0115] In this embodiment, however, the function 
menu is read by asking a device about the usage right 
(discussed later). 

[0116] The function menu 51 may also be made 
directly accessible, without going from the entry in the 
device information list, by predetermining the ID of the 
function menu list or another such method. Here, the 
display parts 52 showing the functions are laid out using 
a display part list, and flags showing the purpose or 
operation of these display parts 52 (for display, for iden- 
tification, for control, etc.), or flags indicating whether 
there is the possibility that these functions will dynami- 
cally disappear, may be added to the entries for the var- 
ious display parts 52. 

[0117] Data objects are laid out in the display part 
list shown by these display part entries. 
[01 1 8] Here, the entries of the display parts 52 may 
be laid out directly in the function menu list, without the 
display part list being used, in which case it is possible 
for the flags indicating purpose and so on to be given in 
the entries of the device information parts 50. The func- 
tion menu list also has information for configuring the 
operation screen of a unit, and the function information 
table 8 has information about the size the screen is sup- 
posed to be, as well as the background color or pattern, 



and other such information as unique information in its 
header or elsewhere. 

[01 19] Meanwhile, the data objects (such as text or 
still pictures) that show this function m nu list its If have 

5 direct entries in the function menu list. 

[0120] Furthermore, a display part list may be used 
for the data objects that show this list itself. The informa- 
tion showing these lists themselves may be distin- 
guished from the display parts 52 or other data objects 

10 by using flags or the like. 

[0121] The display parts 52 are parts used for dis- 
playing icons, buttons, sliders, check boxes, text entries, 
and so forth, and this display part list has unique infor- 
mation such as the type of display parts 52 and inform a- 

15 tion required for each of the display parts 52 (such as 
the variable range, step value, and initial value in the 
case of a slider, for instance). 

[0122] There are entries such as text objects or still 
picture objects in this display part list, and these entries 

20 have unique information about the various data objects 
such as the size of still pictures or flags indicating the 
type (format) of a still picture or text, as well as informa- 
tion about the position on the screen, which is shown as 
a relative position with respect to the menu the device is 

25 supposed to have. 

Layout Information 

[0123] Each display part list contains layout infor- 
30 mation showing the correlation between display parts 
as relationship information having the same information 
(such as the same value) with respect to the display 
parts 52 that are closely related in terms of function or 
screen design. In other words, if the screen size that the 
35 function information table 8 is supposed to have is 
larger than the screen size of the controller, then the 
controller cannot display the display screen shown in 
this function information table 8 as it is, so the controller 
rearranges the layout of the display parts 52 in the func- 
40 tion information table 8 and divides it into a plurality of 
pages for display. 

[0124] In this case, the plurality of closely related 
display parts 52 contain relationship information having 
the same information (value), and belong to a single dis- 
45 play group to be displayed in proximity. As a result of this 
relationship information, the display parts 52 belonging 
to the same display group are laid out in proximity within 
a single page. 

[0125] The "closely related display parts 52" here 
so are, for example, used in pairs for VTR play and stop 
buttons or for the up and down channel buttons on a TV, 
or have a function similar to the cross-shaped buttons 
indicating channels on a TV or aspect switching keys for 
a TV (4-to-3, 16-to-9, letterbox, full-screen display, etc.), 
55 or are laid out in proximity in terms of screen design to 
be more easily understood by the user. 
[0126] The device manufacturer decides independ- 
ently which display parts 52 will be laid out in proximity 
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with this relationship information. This relationship infor- 
mation may also be described in the header as unique 
information. 

[0127] There is no need for this r lationship infor- 
mation to be contained in all of the display parts 52, and s 
is only added to those display parts 52 belonging to one 
of the display groups to be laid out in proximity. 
[0128] Even in the case of a new function that can- 
not be foreseen at present, the data of this new function 
can be transmitted to the controller and information 
related to this new function presented to the user by lay- 
ing out a still picture or the like showing this new func- 
tion as a display part 52. 

[0129] Figure 5 is an explanatory diagram of the 
network control system in this embodiment, and will be 
used to describe the control operations of a controller 
and device. 

[0130] In Figure 5, when a device is connected to 
the transmission line 1 , the controller on this transmis- 
sion line 1 recognizes a new device by bus resetting, in 
the case of a 1394 bus, for instance, and reads the func- 
tion information tables 8 through the transmission line 1 
from the new device as directed by the function informa- 
tion table management means 12, so that these func- 
tion information tables 8 are read and registered in the 
function data base 13 of the controller. 
[0131] Here, the various function information tables 
8 are distinguished, for example, by an ID unique to 
each device, and take on a list structure containing 
entries for the various devices. The function information 
tables 8 may be read all at once here, or only some of 
the function information tables 8 may be read. 
[0132] Furthermore, the reading may be in object 
and list units. 

[0133] If there is not enough memory region in the 
controller here, when a new device is connected, for 
example, the device information parts 50 should be 
read first. In other words, the various parts in the func- 
tion information tables 8 do not necessarily need to 
physically contain the links shown in Figure 5 (associa- 
tion of entries and entities), as long as they can be 
accessed from the controller in object units. 
[0134] For instance, ID for a device information list 
or function menu list may be predetermined, and the 
controller may gain access directly by using this ID, or 
the controller may communicate with the devices so as 
to acquire this ID and gain access. 
[0135] The various display parts 52 are distin- 
guished within the controller by using an ID that com- 
bines the ID unique to a device with the ID of a display 
part 52 attached by a device. 

[01 36] The structure may also be such that the con- 
troller newly reattaches the ID of each display part 52, 
and the controller contains a conversion table of these 
new ID'S and (the unique ID's of the devices + the ID's 
attached by the devices). The display parts 52 contain- 
ing ID's are included in this function information table 8. 
[0137] The display/function selection means 14 



refers to the function information tabl 8 (or part of the 
function information tabl 8) in the function data base 1 3 
through the function information table management 
means 12. 

[0138] When the display/function selection means 
14 displays a list of devices connected to this controller, 
then the display/function selection means 14 uses the 
function information table management means 12 to 
read the data objects (such as text objects or still picture 
objects) in the device information parts 50 from the 
function information tables 8 of all of the devices regis- 
tered in the function data base 13. 
[01 39] The data objects of all of the device informa- 
tion parts 50 do not need to be displayed on the screen, 
and these may be suitably selected and displayed. 
[0140] When there are audio objects among the 
device information parts 50, they are not used in the dis- 
play of a list of devices, and when, for example, a new 
device is connected in a state in which a device list is 
already displayed, then a still picture object of this new 
device is displayed and, if there is an audio object, it is 
played. 

[0141] Next, when the user selects a still picture 
object of a device by means of the pointing function 
(such as the directional key) of a remote control, the dis- 
play/function selection means 14 uses the function 
information table management means 12 to read the 
display parts 52 from the function information list of the 
various functions included in the function menu list, and 
displays the display parts 52 corresponding to the vari- 
ous functions on the screen. As a result it is possible to 
display the display parts 52 showing all of the functions 
of this device. 

[0142] Here again, the identification of the various 
display parts 52 is performed with the ID's unique to the 
devices and the ID's of the various display parts 52. 
[01 43] Since a display part 52 is contained for every 
function here, even when an adequate place for the dis- 
play of the display parts 52 of all functions cannot be 
reserved due to limitations such as the resolution of the 
screen or the effect of other display categories (the dis- 
play of a plurality of devices), the relationship informa- 
tion can be used to divide the display into a plurality of 
pages, affording the flexibility to accommodate such sit- 
uations. 

[0144] When the display screen is small, it is also 
possible to use the text data had by the display parts 52, 
and display only this text data. In other words, since 
each device has a display part 52 for each function, the 
controller can have the final decision right for screen 
display, making possible a more flexible display. 
[01 45] Next, when the user selects a display part 52 
showing the play function of a device, for example, by 
means of the pointing function of a remote control, for 
example, the display/function selection means 14 sends 
the ID of this display part 52 attached by the device as a 
control code to the device along with the operation infor- 
mation of the user (such as "select"). In other words, the 
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ID (control code) of this display part 52 and the opera- 
tion information ("select") of the user are also sent to the 
device when the cursor is moved onto this display part 
52 with the directional key (indicating left, right, down, 
and up) on the remot control, and the select button is 
pressed and then released. 

[0146] It is also possible to send more detailed user 
operation information to a device, and when an opera* 
tion such as "press," "release," or "press twice" is per- 
formed on a display part 52 through operation of a 
remote control of pointing device, it is also possible to 
send this operation information to the device. 
[0147] The user operation information here may be 
encoded and sent along with the ID's of the display 
parts 52, or it may be sent as individual commands (the 
operands are control codes such as the ID's of the dis- 
play parts 52). 

[0148] In this embodiment, with the configuration 
described above, the ID's of the display parts 52 were 
used as control codes, so there is no need to specify 
special control codes such as commands correspond- 
ing to functions with a standardized group or the like, 
and all of the functions had by a device can be control- 
led with a simple configuration. 

[0149] When the user selects a display part 52 indi- 
cating the play function of a device, for example, the dis- 
play/ function selection means 14 displays the playback 
picture from the device on the screen when the proper 
response from the device is returned. In this case, if the 
device wants to change the display part, the device 
transmits the ID of the display part 52 to be changed to 
the controller. Thus, the controller learns that the display 
part 52 of this ID was changed, and retrieves the display 
part 52 of this ID from the device and overwrites the cor- 
responding display part 52. 

[0150] Therefore, when the playback of a VTR has 
been executed, for example, it is possible to perform a 
display showing the playback of the VTR to the user by 
rewriting the display part 52 showing the state of this 
VTR at the directive of the device. 
[0151] When the device wants to change a display 
part here, the data of this display part 52 (including text 
data or still picture data) may be sent to the controller 
along with the ID of the display part 52. Also, the ID, or 
the ID and data, may be sent to the controller in data 
object units rather than display part units. 
[0152] If, for some reason, the device is unable to 
receive a control code sent by the controller, the dis- 
play/function selection means 14 detects that the device 
did not receive the control code based on its response 
to the transmitted control code or the like, and a display 
part 52 independently had by the controller is displayed 
(displayed in a large size in a conspicuous location such 
as the center of the screen, for example), thereby notify- 
ing the user that control failed. 

[0153] Relationship information was used as the 
layout information in this embodiment, but as layout 
information the display parts may mutually have identifi- 



ers of other display parts to be laid out in proximity, and 
if ther are numerous display parts to be laid ut in prox- 
imity, the number of id ntifiers had by the display part 
will increase, but the same effect will be obtain d. 

5 [0154] Furthermore, for a single display group con- 
sisting of a plurality of display parts to be laid out in 
proximity, when each display part has as layout informa- 
tion only one identifier of other display parts to be laid 
out in proximity, and a plurality of display parts are laid 

w out in proximity, the display group to be laid out in prox- 
imity may be indicated by sequentially following the 
identifiers given in the layout information, and while the 
processing of the controller will increase, the same 
effect will be obtained. In this case, the last display part 

15 out of the display group has as layout information an 
identifier indicating that it is the last of the display group, 
instead of an identifier of another display part. 
[01 55] With this embodiment as described above, a 
device has a plurality of display parts that constitute the 

20 operation screen of the device, these display parts con- 
tain layout information indicating the layout relationship 
to be assumed with other display parts (layout informa- 
tion indicating layout in proximity), the controller reads 
the display parts from the device, and the display parts 

25 are displayed on the display screen according to the 
above-mentioned layout information, which means that 
even when only a display screen that is smaller than the 
screen size the device is supposed to have can be used 
with the controller, and the menu of the device cannot 

30 be displayed completely due to limitations such as the 
resolution of the screen or the effect of other display cat- 
egories, an operation screen that is easy for the user to 
understand can still be provided by rearranging the dis- 
play parts according to the layout information or by 

35 dividing the display into a plurality of pages. 

[0156] Also, the layout information is relationship 
information having the same information for a plurality of 
display parts to be laid out in proximity, there is one set 
of layout information for one display group consisting of 

40 a plurality of display parts to be laid out in proximity, and 
the display parts each have this layout information, 
which makes it easy to define the layout information, 
makes it possible to describe the layout information for 
display parts with appended information of a small data 

45 quantity, and makes it possible to describe layout infor- 
mation simply with appended information of a small 
data quantity even when many display parts are to be 
laid out in proximity. 

so Second Embodiment: Relationship information 

[0157] The second embodiment will be described 
using Figures 6 and 7. When a plurality of icons (display 
parts), such as icons representing "play" or "stop," are 
55 displayed on a screen, information about the positional 
relationship in which they will be laid out on the screen 
is stored ahead of time as an attribute of the icons. In 
the second embodiment, relationship information will be 
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described as a specific exampl of layout information. 
[0158] The second embodim nt of the present 
invention will now b described through referenc to the 
figures. 

[0159] Figure 6 is a structural diagram of display 
parts in the second embodiment. Here, everything but 
the display parts are the same as in the first embodi- 
ment, and so will not be described here. 
[0160] In Figure 6, the display parts 52 in this func- 
tion information table 8 have a plurality of sets of rela- 
tionship information indicating the layout information 
about the display parts on the screen. Here, the first set 
of relationship information shall be termed relationship 
information 1 , the second set of relationship information 
relationship information 2, and the third set of relation- 
ship information relationship information 3. The sets of 
relationship information each have different values. 
[0161] Here, when there is only one set of relation- 
ship information, the value thereof is entered in relation- 
ship information 1, and nothing is entered in relationship 
information 2 or 3. In this case, relationship information 
2 and 3 may be omitted, or a null value may be entered. 
Furthermore, flags or the like may be attached to indi- 
cate relationship information and distinguish it from 
other entries or the like. 

[01 62] Similarly, when there are two sets of relation- 
ship information, the values are entered separately in 
relationship information 1 and relationship information 
2, and nothing is entered in relationship information 3. In 
other words, the only significance of relationship infor- 
mation lies in the value, and there is no significance in 
the entry position, whether the entry is in relationship 
information 1 or in relationship information 2. 
[0163] It is also easy to assign priority to the value 
of the relationship information. For instance, a smaller 
value of the relationship information can be specified as 
having a higher priority, and when a controller of a 
screen smaller than the screen size the device is sup- 
posed to have displays these display parts 52, it is pos- 
sible for display parts 52 having relationship information 
with a low value to be displayed on the first page of the 
display screen. 

[0164] Figure 7 is an explanatory diagram illustrat- 
ing the correlation between display parts and relation- 
ship information in this embodiment. 
[0165] Figure 7A shows three display parts 52 of a 
VTR: "play," "stop," and "record." The value of the rela- 
tionship information is "01" for the display part 52 "play," 
"02" for "record," and "01 M and "02" for "stop." 
[0166] In other words, "play" and "stop" are to be 
laid out in proximity, and belong to display group 01. 
"Stop" and "record" are also to be laid out in proximity, 
and belong to display group 02. 

[0167] Figure 7B is an example of when the screen 
size of the controller is the same as or larger than the 
screen size the device is supposed to have in the func- 
tion menu 51 of the function information table 8. In this 
case, the controller displays the display parts 52 on the 



screen as directed by the display parts 52 and the func- 
tion menu 51 in the function information table 8. 
[0168] Figures 7C and 7D are examples of when 
th screen siz of the controller is smaller than the 

5 screen size that the device is supposed to have in the 
function menu 51 of the function information table 8. 
[0169] Here, let us assume that the controller is 
able to display only two buttons (still pictures) within one 
screen. The controller detects from the screen size 

10 information in the function menu 51 or the like that the 
function menu 51 cannot be displayed as directed by 
the device, and divides the display parts 52 into a plural- 
ity of display pages. 

[0170] In this case, relationship information is used 
is in deciding which of the display parts 52 will be 
assigned to which page. In other words, since "play" 
and "stop" belong to display group 01, they should be 
laid out in proximity, and the value of the relationship 
information is smaller compared to the other values. 
20 [0171] As a result, the priority is deemed to be high- 
est for "play" and "stop," and they are displayed on the 
first page, while "stop" and "record" belong to display 
group 02 and should be laid out in proximity, so "stop" 
and "record" are displayed on the next page. The 
25 switching of the pages here is performed with a screen 
scroll key on the remote control, for example. 
[0172] Thus, with this embodiment, the layout infor- 
mation is relationship information having the same infor- 
mation for a plurality of display parts to be laid out in 
30 proximity, and as a result, even when a plurality of dis- 
play groups are present in a single menu, or when there 
are numerous display parts belonging to a display group 
to be laid out in proximity, and even when a plurality of 
display parts make up a plurality of groups, the layout 
35 information can be described simply with appended 
information of small data quantity, and even when only a 
display screen that is smaller than the screen size the 
device is supposed to have can be used with the con- 
troller, and the menu of the device cannot be displayed 
40 completely due to limitations such as the resolution of 
the screen or the effect of other display categories, an 
operation screen that is easy for the user to understand 
can still be provided by rearranging the display parts 
according to the layout information or by dividing the 
45 display into a plurality of pages. 

[0173] Also, if a single display part has a plurality of 
sets of layout information, then even when there are a 
plurality of display parts to be laid out in proximity with 
respect to a single display part, that is, even when a sin- 
so gle display part belongs to a plurality of display groups, 
and even when only a display screen that is smaller 
than the screen size the device is supposed to have can 
be used with the controller, and the menu of the device 
cannot be displayed completely due to limitations such 
55 as the resolution of the screen or the effect of other dis- 
play categories, closely related display parts can still be 
displayed in proximity and always within a single screen 
and an operation screen that is easy for the user to 
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understand and operate can still be provided by display- 
ing a single display part a plurality of times in the divi- 
sion and display of the display parts on a plurality of 
pages according to the layout information. 

Third Embodiment: Identifiers 

[0174] The third embodiment will be described 
using Figure 8. Using a certain icon as a reference, 
information about which icons will be laid out above this 
icon, below it, to the left of it, and so on is contained in 
the attributes of the icon serving as a reference. 
[0175] The third embodiment of the present inven- 
tion will now be described through reference to the fig- 
ures. 

[0176] Figure 8 is a structural diagram of display 
parts in this embodiment. Here, everything but the dis- 
play parts are the same as in the first embodiment, and 
so will not be described here. 

[0177] In Figure 8, the display parts 52 in this func- 
tion information table 8 have identifiers (such as ID) of 
other display parts 52 to be laid out in the four directions 
(up, down, left, and right). 

[0178] Here, the identifiers of the display parts 52 to 
be laid out in proximity in the upward direction are 
entered as upward layout identifiers, and similarly, the 
identifiers of the display parts 52 to be laid out in prox- 
imity in the downward, right, and left directions are 
entered as downward layout identifiers, right layout 
identifiers, and left layout identifiers, respectively. 
[0179] When there is only layout information here in 
the right direction, for example, the identifiers of the dis- 
play parts 52 to be displayed as right layout identifiers 
are entered, but the upward layout identifiers, downward 
layout identifiers, and left layout identifiers are not 
entered. In this case, the upward layout identifiers, 
downward layout identifiers, and left layout identifiers 
may be omitted, or a null value may be entered. 
[0180] Furthermore, flags or the like may be 
attached to indicate relationship information and distin- 
guish it from other entries or the like. 
[0181] Thus, if only a display screen that is smaller 
than the screen size the device is supposed to have can 
be used with the controller, the identifiers of the display 
parts to be laid out in proximity can be used as layout 
information, and the display part can be rearranged 
according to this layout information, or the display can 
be divided into a plurality of pages, which makes it easy 
define the layout information, makes it possible for the 
controller to directly detect the display parts to be laid 
out in proximity, and makes it possible for the display 
parts to be laid out in proximity to be displayed simply 
on the display screen. 

[0182] Furthermore, if there are identifiers for the 
display part to be laid out in proximity in the up, down, 
left, and right directions, the device can specify more 
precisely the display method such as rearrangement or 
division into a plurality of pages. 



[0183] It is also possible to assign a priority to th 
value of th r lationship information. For instance, the 
various layout information can b distinguished with 
flags, entry locations of identifiers showing layout infor- 
mation that ar closer to the header can be specified to 
have a higher priority, and when a controller whose 
screen is smaller than the screen size the device is sup- 
posed to have displays these display parts 52, the dis- 
play parts 52 having layout information in which the 
entry locations of the identifiers showing layout informa- 
tion are closer to the header can be displayed preferen- 
tially, so that they are displayed on the first page of the 
display screen. 

[0184] Thus, with this embodiment, the layout infor- 
mation is identifiers of display parts to be laid out in 
proximity, and as a result, it is easier to define the layout 
information, the controller can directly detect display 
parts to be laid out in proximity, it is possible for display 
parts that are to be laid out in proximity to be displayed 
on the display screen more simply, and even when only 
a display screen that is smaller than the screen size the 
device is supposed to have can be used with the con- 
troller, and the menu of the device cannot be displayed 
completely due to limitations such as the resolution of 
the screen or the effect of other display categories, an 
operation screen that is easy for the user to understand 
can still be provided by rearranging the display parts 
according to the layout information or by dividing the 
display into a plurality of pages. 

[0185] Furthermore, the display parts have identifi- 
ers of display parts to be laid out in proximity in the up, 
down, left, and right directions, and as a result, the 
device can specify the display method more precisely, 
and the intent of the person who created the device 
menu can be conveyed easily to the user even when the 
controller only has a display screen that is smaller than 
the screen size the device is supposed to have. 

Fourth Embodiment: Configuration sets 

[0186] The fourth embodiment will be described 
using Figures 9 and 10. Out of numerous icons, those 
with some connection, such as icons for a video control 
system expressing "play," "stop," or the like, are put 
together and managed as a single group, and other 
icons with some connection, such as icons of the a tele- 
vision control system expressing "channel-up" or "chan- 
nel-down," are put together and managed as a single 
group. 

[01 87] The fourth embodiment of the present inven- 
tion will now be described through reference to the fig- 
ures. 

[0188] Figure 9 is a structural diagram of a function 
menu and configuration set parts illustrating the fourth 
embodiment of the present invention. 
[0189] The device information parts (not shown) 
and display parts (not shown) will not be described here 
because they are the same as in the first embodiment. 
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In this embodiment, the display parts may not contain 
layout information. 

[0190] In Figure 9, the function menu 51 in the func- 
tion information table 8 (not shown) compris s a func- 
tion menu list, and has configuration set entries. These 5 
configuration set entries indicate a configuration set list. 
[0191] Here, the identifiers (such as ID) of the con- 
figuration set list can be determined ahead of time, or 
the controller can acquire the identifiers of the configu- 
ration set list by communication with the device, which 
will allow the controller to directly access the configurer 
tion set list. 

[0192] The configuration set parts 55 indicate a set 
of display parts 52 in the function menu 51 , and are a 
set of display parts 52 (display part A and display part 
B) that are closely related functionally and in terms of 
design. 

[0193] These configuration set parts 55 are config- 
ured using a configuration set list, and this configuration 
set list contains entries for the display parts 52 (display 
part A, display part B, text objects, still picture objects) 
that are used to show this list itself and to configure part 
of the operation screen. 

[0194] The display parts 52 used here to show part 
of the operation screen are laid out using the display 
part list (Figures 1 and 8), and flags indicating the pur- 
pose and operation of these display parts 52 (for dis- 
play, for identification, for control, etc.), or flags 
indicating whether there is the possibility that these 
functions will dynamically disappear, may be added to 
the entries for the various display parts 52. 
[0195] The entries of data objects are laid out in the 
display part list (Figures 1 and 8) shown by these dis- 
play part entries. Here, entries of the various data 
objects (such as text objects or still picture objects) may 
be directly laid out in a configuration set list without the 
use of a display part list, in which case flags indicating 
the purpose, etc., can be provided to the entries of the 
display parts 52. 

[0196] Meanwhile, the data objects showing this 
configuration set list itself (such as text objects or still 
picture objects) have direct entries in the configuration 
set list. A display part list may also be used for the data 
objects that show this list itself. Also, the information 
showing these lists themselves may be distinguished 
from other data objects or from the display parts 52 by 
the use of flags or the like. 

[0197] Furthermore, the configuration set list also 
has information for configuring the operation screen of a 
unit, has position information with respect to the func- 
tion menu 51 or the screen size required to display 
these configuration set parts 55, and has information 
about background color, background pattern, and so on 
as unique information in the header, etc. 
[0198] The configuration set list has information 
indicating priority when the controller displays these 
configuration set parts. For instance, when it is specified 
that a lower value for this priority will be a higher priority, 



and when a controller with a screen that is smaller than 
the screen size the device is supposed to have displays 
the function menu 51 of this device, the configuration 
set parts having a priority with a low value will b dis- 
played on the first pag of the display scr en. 
[0199] As shown in Figure 9, there may be a plural- 
ity of configuration set parts in the function menu 51, 
and there may be display parts 52 which do not belong 
to the configuration set parts. Here, the function menu 

51 serves to show the menu of a VTR, and the configu- 
ration set parts 55 have the display parts 52 of the deck 
component of the VTR, which is a part thereof. 
[0200] The display parts 52 having entries in the 
configuration set list, just as in the first embodiment, are 
parts used for displaying icons, buttons, sliders, check 
boxes, text entries, and so forth, and this display part list 
has unique information such as the type of display parts 

52 and information required for each of the display parts 
52 (such as the variable range, step value, and initial 
value in the case of a slider, for instance). 

[0201] There are entries such as text objects or still 
picture objects in this display part list, and these entries 
have unique information about the various data objects 
such as the size of still pictures or flags indicating the 
type (format) of a still picture or text, as well as informa- 
tion about the position on the screen, which is shown as 
a relative position with respect to the configuration set 
parts that device is supposed to have. 
[0202] Here, the information about position on the 
screen was contained in the entries of the various data 
objects, but may instead be contained in the entries of 
the display part list. 

[0203] Figure 10 is an explanatory diagram of an 
example of the display screen in this embodiment. The 
function menu 51 shown in Figure 9 is an example in 
which the controller screen size is the same as or larger 
than the screen size the device is supposed to have in 
the function menu 51 of the function information table 8. 
[0204] in this case, the controller displays on the 
screen the display parts 52 as directed by the function 
menu 51 of the function information table 8, the config- 
uration set part, and the display parts 52. The data 
objects (still pictures and text data) showing the function 
menu 51 itself are displayed as titles on the screen of 
the controller, but the data objects that show the config- 
uration set parts 55 themselves are not displayed. Fur- 
thermore, it is not displayed that the display parts 52 
belong to the configuration set parts 55, such as by sur- 
rounding the display parts 52 belonging to a single con- 
figuration set part with a box or the like. 
[0205] When it is possible to perform display in 
which it is easier for the user to understand if data 
objects that show the configuration set parts 55 them- 
selves are displayed or if it is displayed that the display 
parts 52 belong to the configuration set parts 55, then 
the configuration set parts 55 or the data objects may 
have flags indicating that these are displayed. 
[0206] Figures 10B and 10C are examples of when 
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the screen size of the controller is smaller than the 
screen size the device is supposed to hav in the func- 
tion menu 51 of the function information table 8. Here, 
we will assume that the controller is able to display in a 
single screen just th configuration s t part that indi- 
cates the VTR deck component. 
[0207] The controller uses the screen size informa- 
tion in the function menu 51, the size of the configura- 
tion set parts among the configuration set parts, and so 
on to detect that the function menu 51 is not displayed 
as directed by the device, but can be displayed in con- 
figuration set part units, and produces and displays a 
page for every configuration set part. 
[0208] In this case, which configuration set part will 
be assigned to which page is determined by the priority 
among the configuration set parts. The controller 
assigns the configuration set parts to pages in the order 
of highest priority first. For instance, as shown in Figure 
10B, the menu of the deck component of the VTR, 
which has a high priority, is displayed on the first page. 
[0209] Here, navigation information conveyed to the 
user by an icon showing "next menu" ("next menu" in 
Figure 10B and "previous menu" in Figure 10C) is inde- 
pendently added by the controller. 
[0210] Thus, when the device displays the menu 
specified by the function menu 51 on a plurality of 
pages, navigation information such as a scroll bar or 
icons for displaying the next page or previous page is 
produced independently by the controller and displayed 
on the display screen. 

[0211] Page up and down keys may be provided to 
a pointing device such as a remote control here, allow- 
ing the displayed page on the operation screen to be 
controlled with these keys. Furthermore, data objects 
(still pictures and text data) showing the configuration 
set parts 55 themselves are used as captions on the 
display screen of the controller. 

[0212] When the user clicks on the icon indicating 
"next page" in the operation screen in Figure 10B, the 
controller detects this operation with the display/func- 
tion selection means 14, and displays on the screen the 
configuration set part (VTR tuner) indicating the next 
priority, for example. Text data and an icon indicating the 
"previous menu" are displayed here independently by 
the controller. While not depicted, the display parts 52 
not belonging to the configuration set parts are dis- 
played on a page after all the configuration set parts. 
The display parts 52 not belonging to the configuration 
set parts may have flags indicating priority, in which 
case even those display parts 52 not belonging to the 
configuration set parts can be displayed with the appro- 
priate priority on the display screen. 
[0213] The configuration set parts contained infor- 
mation about priority in this embodiment, but the identi- 
fiers (such as ID) of the configuration set parts may also 
be used, and a lower value of an identifier can be 
treated as a higher priority, allowing the same effect to 
be obtained. 



[0214] The configuration set parts 55 can also be 
used hierarchically, and ther may be separat configu- 
ration set parts 55 among the configuration set parts 55. 
In this cas it is possible to partition the display parts 52 
5 mor finely, allowing for ven more flexibility in th 
screen size of the controller. 

[0215] Furthermore, just as in embodiments 1, 2, 
and 3, there may be layout information showing the lay- 
out relationship to be assumed by the configuration set 

10 parts 55 with other configuration set parts 55 (layout 
information showing layout in proximity), and even when 
only a display screen that is smaller than the screen size 
the device is supposed to have can be used with the 
controller, but a plurality of configuration set parts can 

15 be displayed, an operation screen that is easy for the 
user to understand and operate can still be provided by 
rearranging the display parts according to the layout 
information or by dividing the display into a plurality of 
pages. 

20 [0216] As discussed above, with the present inven- 
tion, a device has a plurality of display parts that make 
up the operation screen of this device, and configuration 
set parts that indicate a set consisting of a number of 
the above-mentioned display parts, and a controller 

25 reads these display parts and configuration set parts 
from this device and displays the display parts on a dis- 
play screen according to the information of the configu- 
ration set parts, and as a result, even when only a 
display screen that is smaller than the screen size the 

30 device is supposed to have can be used with the con- 
troller, and the menu of the device cannot be displayed 
completely due to limitations such as the resolution of 
the screen or the effect of other display categories, the 
intent of the person who created the menu can be con- 

35 veyed to the user even for controllers of different screen 
sizes. Also, the controller is able to handle the display 
parts and display them on the screen in configuration 
set part units, which simplifies processing. 
[0217] Also, because the various configuration set 

40 parts have information indicating the priority of the dis- 
play, even when the controller only has a display screen 
that is smaller than the screen size the device is sup- 
posed to have, it is possible for the controller to display 
the information on the screen in the order of its impor- 

45 tance to the user, the result of which is a display screen 
that is easier to view. Furthermore, the menu producer 
is able to create an optimal design for a given screen 
size more freely, and is also able to include in the design 
such considerations as how the individual screens 

so appear for different screen sizes, and particularly the 
division of the pages and the relationship between the 
preceding and succeeding pages. 
[0218] Because the configuration set parts have a 
hierarchical structure, the menu producer is able to 

55 assume more screen sizes in the production of the 
device menus, and the controller can display variously 
suitable menu screens for more screen sizes. Further- 
more, the device has a menu consisting of a plurality of 
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display parts showing the operation scr ns of this 
device, the above-mentioned controller reads these dis- 
play parts from this device, and when the size of this 
menu is larger than the display scr en of the controller, 
the controller divides the display into a plurality of pages 5 
the same size or smaller than the above-mentioned dis- 
play screen, and produces information about navigation 
between these plurality of pages and displays this infor- 
mation on the above-mentioned display screen, and as 
a result, even when only a display screen that is smaller 10 
than the screen size the device is supposed to have can 
be used with the controller, and the menu of the device 
cannot be displayed completely due to limitations such 
as the resolution of the screen or the effect of other dis- 
play categories, an operation screen that is easy for the 15 
user to understand can still be provided by dividing the 
display into a plurality of pages. Furthermore, control- 
lers of a variety of display screen sizes can be easily 
accommodated by virtue of the controller having the ini- 
tiative in regard to the production of information about 20 
navigation between these plurality of pages. 

Fifth Embodiment: Operation information 

[021 9] The fifth embodiment will be described using 25 
Figures 1 1 to 13. When the up arrow button, down arrow 
button, right arrow button, or left arrow button provided 
to the controller is pressed, information about which 
icon the selection will switch to from the icon currently 
selected is contained as an attribute of the currently 30 
selected icon. 

[0220] The fifth embodiment of the present inven- 
tion will now be described through reference to the fig- 
ures. 

[0221] Figure 11 is structural diagrams of display 35 
parts illustrating the fifth embodiment of the present 
invention. Here, everything but the display parts are the 
same as in the first embodiment, and will therefore not 
be described. 

[0222] In Figure 11 A, the display parts 52 in the 40 
function information table 8 (not shown) of the device 
have operation information showing such information as 
the switching of the display screen or the movement of 
the cursor within the display screen. With this embodi- 
ment, the device has a plurality of menus, and the oper- 45 
ation information of the display part showing "menu 1" is 
an identifier (such as ID) of a different menu from "menu 
1" to which this display part belongs. Here, the plurality 
of menus have entries as a function menu 51 in the 
device information list or function menu list of Figure 1 , so 
and it is possible to use this information to read from the 
device to the controller. The function information table 8 
does not necessarily have to have the configuration 
shown in Figure 1, and the configuration may instead be 
such that the function menu list showing the menus is 55 
read directly, without following the entries from the func- 
tion menu list or device information list, by predetermin- 
ing the ID'S of a plurality of function menus 51 or making 



an inquiry to the device. 

[0223] When the controller r ads th "main menu," 
it also reads th display parts 52 showing "menu 1" that 
has entries in the function menu 51 that shows this 
"main menu," and the display parts shown in Figure 1 1 A 
are displayed as shown in Figure 1 1 B. Here, the display 
parts 52 other than "menu 1" are the same as the dis- 
play parts 52 shown in Figure 1, but there is no need for 
relationship information. The configuration may also be 
such that the display part list is read individually by, for 
example, first reading only the function menu list, and 
acquiring the ID'S of the display parts 52 given in the 
various entries. 

[0224] When the display parts 52 of "menu 1" have 
been selected in the main menu shown in Figure 1 1 B 
(when the cursor is over these display parts and the 
select key of the remote control 22 is pressed in that 
state), menu 1 shown in Figure 1 1C is displayed on the 
screen of the controller. The displayed menu here is a 
sub-menu that assists the main menu. 
[0225] There is also a help menu that gives expla- 
nations of the functions of the various display parts in 
display part units of the device, and each display part 
has operation information for its respective help menu 
as an identifier for the help menu, providing the user 
with an operation screen that is easy to use and which 
allows the help menu for that display part to be dis- 
played on the screen or an explanation of each function 
and each menu to be displayed when the help button on 
the remote control 22 is pressed while the cursor is on a 
display part. The operation information for the help 
menu is differentiated by flags or the like that show other 
operation information and the help menu operation 
information. 

[0226] Thus, with a simple configuration, the menu 
producer can assume the operation of the user in pro- 
ducing a menu, so the user can be provided with an 
operation screen that is easier to use. Furthermore, if 
the controller reads a plurality of menus ahead of time, 
there will be no need to read the information on the dis- 
play screen from the device every time the user per- 
forms an operation, so the display screen can be 
changed more quickly. 

[0227] With this embodiment, the display screen of 
the controller was displayed by switching, but the main 
menu and menu 1 may also be displayed superimposed 
and menu 1 made active (a state in which menu 1 can 
be operated by the user) by overlap display or the like, 
or the main menu may be reduced in size, and menu 1 
displayed and activated in the resulting empty space. 
[0228] Furthermore, the configuration set parts 
shown by the operation information may be activated 
(put in a state in which the user can operate the display 
parts among the configuration set parts) when the menu 
has configuration set parts and the display parts 52 hav- 
ing the ID'S of the configuration set parts as operation 
information are used to select the icon, in which case 
the same effect is obtained. 
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[0229] Also, when the display parts 52 hav the 
identifiers of other display parts 52 as operation infor- 
mation, and the display parts 52 having operation infor- 
mation ar selected, th cursor may be moved to the 
display parts 52 shown by this operation information. 5 
[0230] For instance, as shown in Figure 12, the dis- 
play parts 52 have operation information indicating cur- 
sor movement in the up, down, left, and right directions 
within the display screen. The operation information 
here is the identifiers (such as ID) of the display parts. 10 
An example of this operation information is the opera- 
tion information in the up, down, left, and right directions 
shown in Figure 13A. Here, the up operation information 
of the display part 52 indicating play is the identifier (ID) 
of the display part 52 itself that indicates "play," the 15 
down operation information is the identifier (ID) of the 
display part 52 that indicates "stop," the left operation 
information is the identifier (ID) of the display part 52 
that indicates "rewind," and the right operation informa- 
tion is the identifier (ID) of the display part 52 that indi- 20 
cates "fast forward." These display parts 52 are 
displayed as in Figure 13B by means of the position 
information included in unique information such as the 
respective headers, and when the cursor (which indi- 
cates the focus of the user) is on the display part 52 25 
showing "rewind," the user uses the remote control 22 
shown in Figure 4, for example, to press the "right but- 
ton" on a cross-shaped directional key indicating the 
directions up, down, left, and right, for example, where- 
upon the cursor moves to the display part 52 showing 30 
"fast forward" according to the operation information. 
Similarly, the cursor moves to the display part 52 show- 
ing "play" when the "up button" is pressed, the cursor 
moves to the display part 52 showing "stop" when the 
"down button" is pressed, and the cursor does not move 35 
when the "left button" is pressed. Thus, with a simple 
configuration, the menu producer can assume the oper- 
ation of the user in producing a menu, so the user can • 
be provided with an operation screen that is easier to 
use. 40 
[0231] The processing for these cursor movements 
is performed by the controller according to the operation 
information directed by the device to the various display 
parts, and there is no need for inquiry to the device. This 
makes it possible for changes in the display screen to 45 
be carried out more quickly. 

[0232] For example, when only a display screen 
that is smaller than the screen size the device is sup- 
posed to have can be used with the controller, and the 
menu of the device cannot be displayed completely, so 
even if the positions of the display parts are rearranged 
by the controller, the user can be simply and clearly 
apprised of the provision of the various functions and 
menus by specifying the movement of the cursor with 
operation information, so an operation screen that is 55 
easy to operate can be provided. 
[0233] It was said here that when the operation 
information is an identifier of a display part, the cursor is 



moved to the display part showing the operation infor- 
mation within the display scr en, but when the display 
parts 52 specified in the operation information are not 
all on the sam page becaus the controller has rear- 
ranged the display parts and divid d them into a plural- 
ity of pages, then the display pages may be updated by 
screen scrolling, screen updating, or the like, the display 
part given in the operation information displayed on one 
page, and the cursor moved to this display part. 
[0234] As discussed above, with this embodiment, 
a device has display parts having operation information 
for changing the display of the operation screen by 
operation from the user, and a menu consisting of a plu- 
rality of display parts showing the operation screen, and 
the controller reads these display parts and configura- 
tion set parts from this device and displays the menu of 
the device on the display screen, and the display of the 
display screen is changed by operation from the user 
according to the operation information, so an operation 
screen that is easy to understand can be provided to the 
user, and since there is no need to read the information 
of the display screen from the device every time the 
user performs an operation, the display screen can be 
changed more quickly. Furthermore, when only a dis- 
play screen that is smaller than the screen size the 
device is supposed to have can be used with the con- 
troller, even if the positions of the various display parts 
have been rearranged by the controller, the user can be 
simply and clearly apprised of the provision of the vari- 
ous functions and menus by defining the movement of 
the cursor with operation information, so an operation 
screen that is easy to operate can be provided. 
[0235] Furthermore, using identifiers showing other 
display parts for the operation intonation makes it sim- 
ple to define the movement of the cursor, and allows the 
provision of the various functions shown by the display 
parts to be presented in a straightforward manner to the 
user, so the user is able to learn of the relationship 
between the functions intuitively, and since there is no 
need to ask the device about cursor movement, opera- 
tion is easier. 

[0236] Also, if the menus have configuration set 
parts, and the operation information is identifiers indi- 
cating configuration set parts, the user can be easily 
apprised of the association between the various config- 
uration set parts, allowing the provision of an operation 
screen that is easier to use. 

[0237] If the device has a plurality of menus, and if 
display parts having as operation information identifiers 
indicating other menus are provided within the menus, 
then menus can be switched with ease, even though 
there is no button for menu switching or the like on the 
remote control, etc. Furthermore, if the device has a 
help menu giving explanations of the functions shown 
by the various devices, and if the display parts have as 
operation information identifiers indicating the help 
menu, then an operation screen that can display expla- 
nations of the various functions and menus with ease 
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and that is easier to use can be provided to the user 
Sixth Embodiment: Overview of channel information 

[0238] The sixth embodiment will be described 
using Figures 14 to 16. While a local display, including 
icons or the like, is being performed, information about 
the video sent to the background thereof is given to the 
attributes of the icons. 

[0239] The sixth embodiment of the present inven- 
tion will now be described through reference to the fig- 
ures. 

[0240] In Figure 14, the display part list for the vari- 
ous display parts 52 that use the transmission line 1 to 
transmit, receive, and process stream data (synchro- 
nous data) such as video and audio data has channel 
information showing the channel numbers for transmit- 
ting synchronous data over the transmission line 1. 
Here, even when a channel for the transmission of asyn- 
chronous data is used, channel information should be 
contained in the display part list of the display parts 52 
showing the functions of transmitting and receiving 
asynchronous data using the transmission line 1. 
[0241] The display part 52 in Figure 14 indicates the 
playing of the VTR, and has a play icon (still picture) and 
text "play*'). The display part list showing this display 
part 52 has channel information (1 channel). This chan- 
nel information indicates a channel that is partitioned by 
a method such as time sharing a band of the transmis- 
sion line 1 that handles synchronous data, and the 
number of this channel is listed as channel information. 
This channel information is used in sending out stream 
data (such as video or audio data) from the device when 
this display part 52, which means play the VTR, has 
been selected by the user. 

[0242] The channel information may be contained 
in the header of the display part list, for example. Chan- 
nel information may also be provided by type of stream 
data, such as video or audio. 

[0243] Figure 1 5 is an explanatory diagram explain- 
ing control related to the function menu in the sixth 
embodiment, and Figure 16 is an explanatory diagram 
explaining control related to the display part. The oper- 
ation of the controller and device will be described using 
these figures. 

[0244] First, when a device is connected to the 
transmission line 1 , the controller on this transmission 
line 1 recognizes a new device by bus resetting, in the 
case of a 1394 bus, for instance, reads the device infor- 
mation parts 50 of the function information table 8 
through the transmission line 1 from the new device as 
directed by the function information table management 
means 12, and reads and registers the device informa- 
tion parts 50 of this function information table 8 in the 
function data base 13 of the controller. The device infor- 
mation parts 50 of the function information table 8 here 
are, for example, as shown in Figure 14, and have text 
objects indicating the manufacturer name, data objects 



indicating the device (still picture objects or text 
objects), and so forth. The various parts in the function 
information table 8 do not necessarily hav to physically 
have the links shown in Figure 14 (association betw en 

5 entries and entities), as long as they can be accessed 
from the controller in the various object units. 
[0245] Here, the various function information tables 
8 in the controller are distinguished by unique ID'S for 
each device, for example, and take on a list structure 

w having an entry for every device. The function informa- 
tion tables 8 may be read ail at once here, or just some 
of the function information tables 8 may be read. They 
may also be read in object or list units. 
[0246] If there is enough memory space in the con- 

15 troller here, the configuration may be such that when a 
new device is connected, for example, links are first fol- 
lowed from the device information parts 50, and the 
function information tables 8 are all read. 
[0247] The display/function selection means 14 

20 refers to the device information parts 50 among the 
function information tables 8 in the function data base 
13 through the function information table management 
means 12. 

[0248] When the display/function selection means 

25 14 displays a list of the devices connected to this con- 
troller, the display/function selection means 14 uses the 
function information table management means 12 to 
read the data objects (such as text objects or still picture 
objects) among the device information parts 50 from all 

30 of the function information tables 8 registered in the 
function data base 13, and displays them on the screen. 
[0249] Not all of the data objects among the device 
information parts 50 need to be displayed on the 
screen, and may be selected and displayed as needed. 

35 [0250] Here, if there are audio objects among the 
device information parts 50, they are not used in the dis- 
play of the list of devices; for example, when a new 
device is connected in a state in which a list of devices 
has already been displayed, the still picture object of 

40 this new device is displayed and, if there is an audio 
object, it is played. Alternatively, it is played when the 
device information parts 50 are selected by the user. 
[0251] When, for example, the user selects the still 
picture object of a device using the pointing function 

45 (such as a directional key) of a remote control, the dis- 
play/function selection means 14 notifies the function 
information table management means 12 that the still 
picture of the device has been selected, and requests 
the function menu 51 for that device. 

so [0252] The control related to the function menu will 
now be described using Figure 15. 
[0253] If there is no function menu for that device 
here, then the function information table management 
means 12 uses the controller asynchronous data 

55 processing means 11 to request the right to use that 
device, as shown in Figure 15. If use is granted for the 
device for which a usage right has been requested, the 
identifier (such as a unique ID) of that controller is 
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stored in the function information management means 
1 7 of the device, and a menu list (function menu list) 
containing the configuration of the menus showing the 
functions of the device is fetched from the function infor- 
mation table 8 via the function information management 
means and sent to the controller. The controller uses 
the function information table management means 12 to 
store this function menu list in the function information 
table 8 indicating this device. Here, a list of the display 
parts 52 included in the menu may be displayed, or just 
a list of the identifiers (ID'S) of the display parts 52 or the 
like may be displayed. 

[0254] Meanwhile, when no usage right is granted, 
the device returns error information, and the controller 
displays on the screen that this device cannot be used, 
thereby notifying the user. 

[0255] Permission for the usage right to a device 
here means permitting a controller to control that 
device, but may also mean just permitting the display of 
the function menu, or may mean granting an exclusive 
right to the device. 

[0256] Furthermore, the function information table 
management means 12 fetches the identifiers (ID's) of 
the display parts 52 belonging to the function menu list 
from the stored function menu list from the entry infor- 
mation for the display parts 52 and so on. The function 
information table management means 1 2 requests text 
objects, still picture objects, or display parts 52 belong- 
ing to the function menu 51 using the controller asyn- 
chronous data processing means 11 and obtains the 
display parts 52 as the response to this, and the func- 
tion information table management means 12 stores 
these in the function information tables 8. Here, the 
function information tables 8 in the controller do not 
need to be in the same format as the function informa- 
tion tables 8 in the device, but should include the infor- 
mation of the function information tables 8 in the device. 
[0257] Using the function information table man- 
agement means 12, the display parts 52 of the various 
functions listed in the function menu list are read out 
from the function information table 8 of this device, and 
all the display parts 52 corresponding to the respective 
functions are displayed on the screen. By repeating this 
for all of the display parts 52 belonging to the function 
menu 51 , it is possible to display the display parts 52 
showing all of the functions of this device. 
[0258] Here, the distinction between the various 
display parts 52 in the controller is performed with the 
unique ID of the device and the ID'S of the display parts 
52 attached by the device. 

[0259] The configuration may also be such that the 
controller newly reattaches the ID's of the display parts 
52, and the controller has a conversion table for these 
new ID*s and (the unique ID of the device + the ID'S 
attached to the device). The display parts 52 containing 
ID's are included in this function information table 8. 
[0260] When the user selects a certain display part 
52, the control code (ID) of this display part 52 and the 



operation information from the user are sent by the con- 
troller to the device. Here, when any display part 52 
belonging to th function information tabl 8 chang s, 
the device sends th changed display part 52 to the 
5 controller in r sponse to the transmission from the con- 
troller. In this case, the device may return the ID of the 
display part 52 to the controller, and if needed, the con- 
troller may read and display this display part 52 from the 
device. 

10 [0261] When the contents of the display parts 52 of 
the function information table 8 in the device have spon- 
taneously changed, such as when the device has 
changed the still picture of the display parts 52 showing 
the functions of a VTR deck because the tape has been 

15 removed from the VTR and the deck can no longer be 
operated (for instance, when the still pictures of the var- 
ious functions are replaced with still pictures to which an 
"X" has been added in order to show that the functions 
will not work), the device spontaneously sends the 

20 changed display parts 52 to the controller having the 
usage right. 

. [0262] When the user then selects a display part 
52, the control code (ID) of this display part 52 and the 
operation information from the user are sent by the con- 
25 trailer to the device, and the response is similarly proc- 
essed. Repeating this allows the device to be controlled 
from the controller. 

[0263] When the controller stops using this device, 
the controller notifies the device that it is relinquishing 

30 its usage right, and the use of the device ends. At this 
point, at the same time, the channel of the transmission 
line 1 that was in use at the directive of this controller is 
released by the device. Thus, the controller is prevented 
from disappearing while the transmission line 1 is still 

35 being used, which allows the transmission line 1 to be 
utilized more efficiently. 

[0264] The operation when the user selects the dis- 
play parts 52 will now be described using Figure 16. 
[0265] When the user selects a* display part 52 

40 showing the play function of the device, for example, 
using the pointing function of a remote control or the 
like, the display/function selection means 14 sends the 
ID (such as X1) of this display parts 52 attached by the 
device as a control code to the device along with the 

45 operation information from the user (such as "select"). 
In other words, when the cursor is moved over this dis- 
play part 52 with the four-way directional key of a remote 
control, and the select button is pressed and then 
released, the ID (control code: X1) of this display part 

so 52 and the operation information from the user ("select") 
are sent to the device. 

[0266] It is also possible to send more detailed user 
operation information to the device, and when opera- 
tions such as "press," "release," "press twice" are per- 
55 formed on a display part 52 by operation of a remote 
control or pointing device, it is also possible to send this 
operation information to the device. 
[0267] Also, rather than sending the user operation 
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information to the device, just the control code of the 
display part 52 (the identifier of the data object or the 
display part, for example) may be transmitted to the 
d vice, which allows the configuration to be simplified. 
[0268] The operation information from the user here 
may be encoded and sent along with the ID of the dis- 
play part 52, or each may be sent as a single command 
(the operand being a control code such as the ID of the 
display part 52). 

[0269] The device receives the control code (ID: 
X1) of the display part 52 selected on the controller and 
the operation information from the user ("select"), inter- 
prets the meaning of this received data with the device 
asynchronous data processing means 6, determines 
that the VTR is to be played, and outputs a directive of 
play to the unit internal control means 9. The unit inter- 
nal control means 9 at this time reserves a band of the 
transmission line 1 required for the play of the VTR, and 
acquires a channel as a predetermined value. The play 
operation of the VTR is then started, and stream data is 
outputted to a channel (CH 1) of the transmission line 1. 
[0270] Since the ID of the display part 52 was used 
here as a control code, there is no need for a special 
control code such as a command corresponding to a 
function to be specified with a standardized group or the 
like, and all of the functions had by the device can be 
controlled with a simple configuration. 
[0271] After the controller gets a response from the 
device and the correct response is returned, the chan- 
nel of the transmission line 1 containing the channel 
information (CH 1) is set to receive, the packet of stream 
data flowing through channel 1 of the transmission line 
1 is read, and decoding and other such processing is 
performed by the controller signal processing means 10 
for display on the display screen. 
[0272] For instance, when the user selects the dis- 
play part 52 showing the play function of the device, the 
display/function selection means 14 displays the play 
picture from the device on the screen when the correct 
response has been returned from the device. In this 
case, if the device wants to change the display part, the 
device transmits the display part 52 to be changed 
(including text data or still picture data) to the controller. 
Thus, the controller learns that the display part 52 has 
been changed, and overwrites the corresponding dis- 
play part 52. For example, when the VTR is played, the 
display part 52 indicating this play state of the VTR 
(such as one in which the color of the still picture show- 
ing play has been inverted) is overwritten on the display 
part showing play by the directive of the device, which 
makes it possible to perform display that shows the user 
that the VTR is playing. 

[0273] Here, a function menu showing the functions 
of the device is displayed on part of the display screen 
of the controller in the display of the stream data on the 
display screen of the controller. In the display of the 
stream data, the configuration may be such that the 
function menu 51 is not displayed, but is redisplayed on 



the screen when, for example, the user presses the 
menu key of the remote control 22, and since the stream 
data and the function menu 51 ar not displayed on the 
display sere n at th sam tim , th user is able to 

5 enjoy the playback of th stream data. Also, in the dis- 
play of the stream data, the stream data may be dis- 
played as a background for the function menu 51 . 
[0274] When the device wants to change a display 
part here, the device may transmit the ID of a display 

10 part 52 rather than the display part 52, in which case the 
controller may use this ID to read the data of the display 
part 52 (including text data or still picture data). Also, the 
ID alone, or the ID and data, may be transmitted to the 
controller not in display part units but in data object 

15 units. 

[0275] If the device was for some reason not able to 
receive the control code transmitted by the controller, 
then the display/function selection means 14 detects 
that the device has not received the control code by 

20 using the response to the transmitted control code or 
the like, and a display part 52 independently had by the 
controller is displayed (displayed in a large size in a con- 
spicuous location such as the center of the screen, for 
example), thereby notifying the user that control failed. 

25 The same applies when the controller transmits stream 
data, and the device receives it. 
[0276] With this embodiment, the controller 
received a stream flowing in a channel of the transmis- 
sion line 1, but it is also possible for the controller to 

30 direct that the stream data flowing through this channel 
be received by another device or controller, which 
makes it easy to display stream data on a plurality of 
screens or to copy what is being played, allows the con- 
troller to control the flow of stream data simply and 

35 freely, and allows more functions to be provided to the 
user. 

[0277] Also, the stream data flowing through the 
channel in the transmission line 1 was video and audio 
data here, but may instead be just video or just audio, or 

40 other data (character data or print data used with a 
printer), and in more specific terms, may be MPEG or 
DV data, and as long as the function shown by a display 
part 52 transfers data over the transmission line 1 using 
a channel, any such data may be used as stream data. 

45 [0278] Furthermore, if the display parts 52 have 
flags indicating the type of this stream data, stream data 
that cannot be handled by the controller can be 
detected before the controller displays this display part 
on the screen, and it is possible to notify the user that 

so the function shown by this display part 52 cannot be 
played and displayed by this controller, which means 
that an operation screen which is easier to use can be 
provided. 

[0279] Furthermore, the menu was shown for a sin- 
55 gle device in this embodiment, but it is also possible for 
the controller to gather display parts from a plurality of 
devices and to construct and display an operation 
screen using these display parts, in which case the fact 
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that the various display parts have channel information 
makes it possibl for the controller to detect whether the 
functions shown by the various display parts use the 
same channel. For instance, it is possible for the func- 
tion shown by a certain display part to be display d 5 
such that other display parts can be selected during 
execution, allowing visual recognition by the user to be 
enhanced. 

[0280] Thus, if there is channel information for 
every display part 52, the controller can simultaneously 
execute a plurality of functions, making it possible to dis- 
play a plurality of streams simultaneously on the control- 
ler screen (such as watching two programs at the same 
time), to watch one program while recording another, 
and so on. It is also possible for the controller to receive 
and display on the display screen the channels on the 
transmission line 1 given in the channel information 
even when another controller is controlling the device in 
question, so the user can be notified of the status of the 
unit being used by the other controller by means of 
played video/audio or the like on the display screen, and 
an operation screen that is easier for the user to under- 
stand can be constructed. 

[0281] The device determined the channel informa- 
tion ahead of time in this embodiment, but after the 
device has received a control code from the controller, a 
band and channel may be reserved on the transmission 
line 1 , and this channel information sent to the control- 
ler. For example, the configuration may be such that 
when the user presses the still picture (such as a play 
button) of a display part 52 having channel information 
showing that one of the channels on the transmission 
line 1 is being used (that is, information showing that a 
channel has been determined after selection of this 
unit), the device receives the control code of this still pic- 
ture (such as a play button), the device acquires a band 
and channel after receipt of a control code and a still 
picture showing that this play button has been pressed 
(such as one in which the color of the still picture show- 
ing the play button has been inverted), and the display 
part 52 including this channel information is sent to the 
controller. 

[0282] Also, the device may enter the channel infor- 
mation scheduled for use in a display part 52, and after 
the device receives the control code of this display part 
52 from the controller, the device may reserve a band 
and channel and send the specified channel information 
as a control code response to the controller. 
[0283] Furthermore, the configuration may be such 
that the channel information is rewritable, the device 
reserves a band and channel while another controller is 
using the device, for example, the channel information 
of this display part 52 in the function information table 8 
is updated as needed, and the controller is notified that 
this display part has been changed. In this case, a 
change in the status of a device (such as the acquired 
channel of the device) by another controller can also be 
easily accommodated by displaying the output stream 



ofthedevic . 

[0284] As discussed above, with this embodiment, 
the device has display parts which make up the opera- 
tion screen of the device and contain channel informa- 
tion that shows the chann I of the transmission lin , the 
controller reads the display parts from the device and 
sets the channel of the transmission line according to 
the channel information when handling data shown by 
the display parts, which makes it possible to share 
channel information between the controller and the 
device with a simple configuration, allowing the control- 
ler to control the flow of stream data with ease. Also, a 
plurality of data, streams can be executed simultane- 
ously if there is channel information for every display 
part with a device that can handle a plurality of data 
streams simultaneously. 

[0285] The data shown by the display parts here 
also includes commands, messages, and so forth. 
[0286] If a display part has a flag indicating the type 
of stream, stream data that cannot be handled by the 
controller can be detected before the controller displays 
this display part, and it is possible to notify the user that 
the function shown by this display part 52 cannot be 
used with this controller, which means that an operation 
screen which is easier to use can be provided. 
[0287] When a device has display parts that make 
up the operation screen of the device, and menus com- 
posed of one or more display parts and showing the 
functions of the device, and when the controller 
requests a usage right for a device and the device 
grants this usage right, the device sends the controller a 
menu list including identifiers for the display parts that 
make up the above-mentioned menus of the device, 
which allows the controller to acquire the menus in the 
device by a simple procedure, allows the number of dis- 
play parts in a menu list and so on to be confirmed even 
when there is little memory space in the controller, 
makes it possible to read just what is needed, and 
allows the menus of the device to be displayed more 
efficiently. 

[0288] When there is a change in the display parts 
in a device after the controller has requested the usage 
right for that device, the device can easily ascertain the 
controller having the usage right by sending the above- 
mentioned changed display parts to the controller, mak- 
ing it possible for a notification of a change in the display 
parts in a device to be made efficiently when there is 
such a change. 

[0289] When the controller relinquishes its usage 
right to the device, the device is notified that the usage 
right has been relinquished, and the channel of the 
transmission line that was in use at the directive of this 
controller is released by the device, which prevents the 
device from remaining in a state in which the channel of 
the transmission line is in use, and allows the transmis- 
sion line to be utilized more effectively. 
[0290] When the device releases the usage right, 
the functions of the device may be put in a standby sta- 



ts 



20 



25 



30 



35 



40 



45 



50 



22 



43 



EP 0 996 307 A1 



44 



tus. For instance, with a VTR, play may be stopped. 
Seventh Embodim nt: Channel information 

[0291] The seventh embodiment will be described 5 
using Figures 17 to 19. While a local display, including 
icons or the like, is being performed, information about 
the video sent to the background thereof is given to the 
attributes of the icons. 

[0292] The seventh embodiment of the. present 
invention will now be described through reference to the 
figures. 

[0293] Figure 17 is a structural diagram of the func- 
tion menu and configuration set parts in the seventh 
embodiment. 

[0294] Here, the device information parts (not 
shown) and display parts (not shown) will not be 
described since they are the same as in the first embod- 
iment illustrated in Figure 14. The display parts need not 
contain channel information in this embodiment. 
[0295] In Figure 17, the function menu 51 in the 
function information tables 8 (not shown) consists of a 
function menu list, and has configuration set entries. 
These configuration set entries indicate the configura- 
tion set list. 

[0296] Here, the configuration set list may be made 
directly accessible by the controller by predetermining 
the identifiers (such as ID's) of the configuration set list 
or by having the controller acquire the identifiers of the 
configuration set list by communication between the 
controller and the device. 

[0297] The configuration set parts 55 show sets of 
display parts 52 in the function menu 51 , and are sets of 
display parts 52 (display part A and display part B) that 
are closely related from the standpoint of function or 
design. 

[0298] These configuration set parts 55 are config- 
ured using a configuration set list, and this configuration 
set list contains entries for display parts 52 (display part 
A, display part B, and text objects) for configuring part of 
the operation screen and for showing this list itself. Fur- 
thermore, there may be channel information (CH 5). 
Here, the channel information may be provided in the 
header of the configuration set list, for example. 
[0299] The display parts 52 used here to show part 
of the operation screen are laid out using the display 
part list (Figure 14), and flags showing the purpose or 
operation of these display parts 52 (for display, for iden- 
tification, for control, etc.), or flags indicating whether 
there is the possibility that these functions will dynami- 
cally disappear, may be added to the entries for the var- 
ious display parts 52. The display part list here does not 
contain channel information. 

[0300] Data object entries are laid out in the display 
part list (Figure 14) shown by these display part entries. 
Here, the entries of the various data objects (text 
objects and still picture objects) may be laid out directly 
in the function menu list, without the display part list 



being used, in which case it is possibl for the flags indi- 
cating purpose and so on to be given in the entries of 
the display parts 52. 

[0301] Meanwhile, the data objects (such as text or 
still pictures) that show this function menu list itself have 
direct entries in the function menu list. Furthermore, a 
display part list may be used for the data objects that 
show this list itself. The information showing these lists 
themselves may be distinguished from the display parts 
52 or other data objects by using flags or the like. 
[0302] Furthermore, the configuration set list has 
information for configuring the operation screen of the 
unit, has positional information with respect to the func- 
tion menu 51 or screen size needed for displaying these 
configuration set parts 55, and has information about 
background color, background pattern, and so on as 
unique information in the header, etc. 
[0303] The configuration set list has information in 
the header indicating priority when the controller dis- 
plays these configuration set parts. For instance, when 
it is specified that a lower value for this priority will be a 
higher priority, and when a controller with a screen that 
is smaller than the screen size the device is supposed 
to have displays the function menu 51 of this device, the 
configuration set parts having a priority with a low value 
will be displayed on the first page of the display screen. 
[0304] As shown in Figure 17, there may be a plu- 
rality of configuration set parts in the function menu 51, 
and there may be display parts 52 which do not belong 
to the configuration set parts. Here, the function menu 

51 serves to show the menu of a VTR, and the configu- 
ration set parts 55 have the display parts 52 of the deck 
component of the VTR, which is a part thereof. 
[0305] The display parts 52 having entries in the 
configuration set list, just as in the first embodiment, are 
parts used for displaying icons, buttons, sliders, check 
boxes, text entries, and so forth, and this display part list 
has unique information such as the type of display parts 

52 and information required for each of the display parts 
52 (such as the variable range, step value, and initial 
value). 

[0306] There are entries such as text objects or still 
picture objects in this display part list, and these entries 
have unique information about the various data objects 
such as the size of still pictures or flags indicating the 
type (format) of a still picture or text, as well as informa- 
tion about the position on the screen, which is shown as 
a relative position with respect to the configuration set 
parts that device is supposed to have. 
[0307] Here, the information about position on the 
screen was contained in the entries of the various data 
objects, but may instead be contained in the entries of 
the display part list. 

[0308] The display parts 52 belonging to the config- 
uration set parts 55 do not have channel information 
here, but when these display parts 52 are functions that 
use a channel of th transmission line 1 to transmit 
stream data, then the channel information (CH 5) had 
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by the configuration set parts 55 is used, and this chan- 
nel information is shared betwe n the controller and the 
device. 

[0309] The function menu 51 shown in Figure 17 is 
as shown in Figure 1 0, which is an example in which the 
controller screen size is the same as or larger than the 
screen size the device is supposed to have in the func- 
tion menu 51 of the function information table 8. 
[0310] Here, the reading and control of the device 
information parts 50 and the display parts 52 are the 
same as in the first embodiment, and so will not be 
described again. 

[0311] The control related to the function menu 51 
will be described. 

[0312] When there is no function menu for a device 
here, that is, when the controller has yet to acquire the 
usage right for the device, then the function information 
table management means 12 uses the controller asyn- 
chronous data processing means 1 1 to request a usage 
right from the device, as shown in Figure 18. If the 
device for which a usage right has been requested 
grants this request, then the identifier (such as a unique 
ID) of the controller in question is stored in the function 
information management means 17 of the device, and 
the menu list (function menu list) giving the configura- 
tion of the menus showing the functions of the device is 
fetched from the function information table 8 via the 
function information management means and sent to 
the controller. The controller uses the function informa- 
tion table management means 12 to store this function 
menu list in the function information table 8 showing this 
device. Here, the menu list may be a listing of the con- 
figuration set parts 55 included in the menu and display 
parts 52 not belonging to the configuration set parts 55, 
or may just be a list of identifiers (ID's) of the configura- 
tion set parts 55 and display parts 52, for example. 
[0313] Meanwhile, when a usage right is not 
granted, the device returns error information, and the 
controller displays on the screen that this device cannot 
be used, thereby notifying the user. 
[0314] Permission for the usage right to a device 
here means permitting a controller to control that 
device, but may also mean just permitting the display of 
the function menu, or may mean granting an exclusive 
right to the device. 

[0315] Furthermore, the function information table 
management means 12 fetches the identifiers (ID's) of 
the configuration set parts 55 and the display parts 52 
not belonging to the function menu list from the stored 
function menu list from the entry information for the con- 
figuration set parts 55, the display parts 52, and so on. 
The function information table management means 1 2 
requests text objects, still picture objects, configuration 
set parts 55, or display parts 52 not belonging to the 
configuration set parts 55 using the controller asynchro- 
nous data processing means 1 1 and obtains the config- 
uration set parts 55 and the display parts 52 as the 
response to this, and the function information table 



management means 12 stor s thes in the function 
information tabl s 8. Her , the function information table 
8 in the controller do not need to b in the same format 
as the function information tables 8 in the devic , but 
5 should include the information of the function informa- 
tion table 8 in the device. The various configuration set 
parts 55 and the various display parts 52 not belonging 
to the configuration set parts 55 may be read in any 
order. 

w [0316] The identifiers (ID's) of the display parts 52 
belonging to the function menu list from the function 
menu list stored in the controller are fetched from the 
entry information of the display parts 52, etc. The func- 
tion information table management means 12 requests 

is text objects, still picture objects, or display parts 52 
belonging to the configuration set parts 55 using the 
controller asynchronous data processing means 11, 
obtains the display parts 52 as the response to this, and 
repeats this for all the display parts 52 belonging to the 

20 configuration set parts 55, and the function information 
table management means 12 stores these in the func- 
tion information tables 8. 

[0317] Next, using the function information table 
management means 12, the display parts 52 of various 

25 functions listed in the function menu list are read out 
from the function information tables 8 of this device, and 
all of the display parts 52 are displayed on the screen. 
By repeating this for all of the display parts 52 belonging 
to the function menu 51 , it is possible to display display 

30 parts 52 showing ail of the functions of this device. The 
configuration here may also be such that all of the con- 
figuration set parts 55 and display parts 52 belonging to 
. the function menu 51 are read at once. 
[0318] When the user selects a certain display part 

35 52, the control code (ID) of this display part 52 and the 
operation information from the user are sent by the con- 
troller to the device. Here, when any display part 52 
belonging to the function information table 8 changes, 
the device sends the changed display part 52 to the 

40 controller in response to the transmission from the con- 
troller. In this case, the device may return the ID of the 
display part 52 to the controller, and if needed, the con- 
troller may read and display this display part 52 from the 
device. 

45 [0319] When the user then selects a certain display 
part 52, the control code (ID) of this display part 52 and 
the operation information from the user are sent by the 
controller to the device, and the response is similarly 
processed. Repeating this allows the device to be con- 
so trolled from the controller. 

[0320] When the controller stops using this device, 
the controller notifies the device that it is relinquishing 
its usage right, and the use of the device ends. At this 
point, at the same time, the channel of the transmission 
55 line 1 that was in use at the directive of this controller is 
released by the device. Thus, the controller is prevented 
from disappearing while the transmission line 1 is still 
being used, which allows the transmission line 1 to be 
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utilized more efficiently. 

[0321] Screen display on the controller will now be 
described. 

[0322] Th controller displays the various display 
parts 52 on the screen as directed by the function menu 
51 of the function information table 8, the configuration 
set part, and the display parts 52. The data objects (still 
pictures and text data) showing the function menu 51 
itself are displayed as titles on the screen of the control- 
ler, but the data objects that show the configuration set 
parts 55 themselves are not displayed. Furthermore, it 
is not displayed that the display parts 52 belong to the 
configuration set parts 55, such as by surrounding the 
display parts 52 belonging to a single configuration set 
part with a box or the like. 

[0323] When it is possible to perform display in 
which it is easier for the user to understand if data 
objects that show the configuration set parts 55 them- 
selves are displayed or if it is displayed that the display 
parts 52 belong to the configuration set parts 55, then 
the configuration set parts 55 or the data objects should 
have flags indicating that these are displayed. 
[0324] Here, the display parts 52 indicated by 
"rewind," "play," "fast forward," "pause," "stop," and 
"record" belong to the configuration set part 55 indi- 
cated by "VTR: deck," while the display parts 52 indi- 
cated by "channel up" and "channel down" belong to the 
configuration set part indicated by "VTR: tuner." The 
configuration set parts indicated by "VTR: deck" and 
"VTR: tuner" have their own respective channel informa- 
tion (CH 5 and CH 6). Thus, among the display parts 52 
shown in Figure 10A, out of "rewind," "play," "fast for- 
ward," "pause," "stop," and "record," the functions other 
than "stop," which does not handle stream data, use 
channel information CH 5, while the display parts 52 
indicated by "channel up" and "channel down" use CH 
6. 

[0325] Here, when the controller displays stream 
data, it may be displayed along with the menus that pro- 
duce the device, or only the stream data may be dis- 
played and the menus displayed once again by 
operation of the remote control 22 on the part of the 
user. 

[0326] Therefore, it is possible for different channel 
information to be contained in various function units. For 
example, it is possible for deck playback to be per- 
formed simultaneously while a tuner program is 
watched on a different channel. 

[0327] Figures 10B and 10C are examples of when 
the screen size of the controller is smaller than the 
screen size the device is supposed to have in the func- 
tion menu 51 of the function information table 8. Here, 
we will assume that the controller is able to display in a 
single screen just the configuration set part that indi- 
cates the VTR deck component. 
[0328] We will assume here that it is possible here 
for the display screen of the controller to be in various 
configuration set part units, but display in function menu 



units is impossible. 

[0329] Here, the reading and control of the device 
information parts 50 and the display parts 52 are the 
same as in th first embodim nt, and so will not be 

5 described again. 

[0330] When there is no function menu for a device 
on the controller here, that is, when the controller has 
yet to acquire the usage right for the device, then the 
function information table management means 12 uses 

w the controller asynchronous data processing means 1 1 
to request a usage right from the device, as shown in 
Figure 18. If the device for which a usage right has been 
requested grants this request, then the identifier (such 
as a unique ID) of the controller in question is stored in 

15 the function information management means 17 of the 
device, and the menu list (function menu list) giving the 
configuration of the menus showing the functions of the 
device is fetched from the function information table 8 
via the function information management means and 

20 sent to the controller. The controller uses the function 
information table management means 12 to store this 
function menu list in the function information table 8 
showing this device. Here, the menu list may be a listing 
of the configuration set parts 55 included in the menu 

25 and display parts 52 not belonging to the configuration 
set parts 55, or may just be a list of identifiers (ID's) of 
the configuration set parts 55 and display parts 52, for 
example. 

[0331] Meanwhile, when a usage right cannot 
30 granted, the device returns error information, and the 

controller displays on the screen that this device cannot 

be used, thereby notifying the user. 

[0332] Permission for the usage right to a device 

here means permitting a controller to control that 
35 device, but may also mean just permitting the display of 

the function menu, or may mean granting an exclusive 

right to the device. 

[0333] The controller then reads the configuration 
set list showing the configuration set parts 55, as shown 

40 in Figure 18. Thus, the controller detects that the func- 
tion menu 51 cannot be displayed as directed by the 
device, but that display is possible in configuration set 
part units, which it detects from the screen size informa- 
tion given in the header of the function menu list in the 

45 function menu 51 , from the size of the configuration set 
part given in the header of the configuration set list in 
the configuration set parts 55, or the like. 
[0334] Therefore, the function information table 
management means 12 fetches from the stored function 

so menu list the identifier (ID) of the display part 52 belong- 
ing to the configuration set parts 55 with the highest pri- 
ority given in the header of the configuration set list out 
of the configuration set parts 55 belonging to the func- 
tion menu list, requests text objects, still picture objects, 

55 or display parts 52 belonging to the configuration set 
parts 55 using the controller asynchronous data 
processing means 1 1 and obtains the display parts 52 
as the response to this, and the function information 
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table management means 12 stor s th se in the func- 
tion information tables 8. H re, the function information 
tables 8 in the controller do not ne d to be in the same 
format as the function information tables 8 in th device, 
but should include the information of the function infor- 
mation tables 8 in the device. 

[0335] Next, the function information table manage- 
ment means 12 is used to read the display parts 52 of 
various functions listed in the configuration set list from 
the function information tables 8 of this device, and dis- 
play all of the display parts 52 on the screen. By repeat- 
ing this for all of the display parts 52 belonging to the 
function menu 51, it is possible to display on the screen 
the display parts 52 showing all of the functions 
included in the function units shown by the configuration 
set part. In other words, the controller reads and dis- 
plays the information in the function information tables 8 
of the device in configuration set part units 55. 
[0336] Thus, when the controller switches the dis- 
play screen to the next screen, the above processing is 
repeated and the configuration set part 55 having the 
next priority is displayed. At this point, the information of 
the previous configuration set part 55 may be deleted 
. from the controller, which allows the controller to com- 
prise less memory space. 

[0337] When the user selects a certain display part 
52, the control code (ID) of this display part 52 and the 
operation information from the user are sent by the con- 
troller to the device. Here, when any display part 52 
belonging to the function information table 8 changes, 
the device returns the changed display part 52 to the 
controller in response to the transmission from the con- 
troller. In this case, the device may return the ID of the 
display part 52 to the controller, and if needed, the con- 
troller may read and display this display part 52 from the 
device. 

[0338] When the user then selects a display part 
52, the control code (ID) of this display part 52 and the 
operation information from the user are sent by the con- 
troller to the device, and the response is similarly proc- 
essed. Repeating this allows the device to be controlled 
from the controller. 

[0339] When the controller stops using this device, 
the controller notifies the .device that it is relinquishing 
its usage right, and the use of the device ends. At this 
point, at the same time, the channel of the transmission 
line 1 that was in use at the directive of this controller is 
released by the device. Thus, the controller is prevented 
from disappearing while the transmission line 1 is still 
being used, which allows the transmission line 1 to be 
utilized more efficiently. 

[0340] Screen display on the controller will now be 
described. 

[0341] The controller detects that the function menu 
51 cannot be displayed as directed by the device, but 
that display is possible in configuration set part units, 
which it detects from the screen size information given 
in the header of the function menu list in the function 



menu 51, from th size of the configuration set part 
given in the header of the configuration set list in the 
configuration set parts 55, or th like, and produces and 
displays a page for every configuration set part. 

5 [0342] In this case, the decision as to which config- 
uration set part will be assigned to which page is made 
by the priority given in the header of the configuration 
set parts 55. The controller assigns the configuration 
set parts to their respective configuration set parts start- 

10 ing with the one with the highest priority. For instance, 
as shown in Figure 10B, the menu for the VTR deck, 
which has a high priority, is displayed on the first page. 
[0343] Here, navigation information conveyed to the 
user by an icon showing "next menu" ("next menu" in 

15 Figure 10B and "previous menu" in Figure 10C) is inde- 
pendently added by the controller. 
[0344] Thus, when the device displays the menu 
specified by the function menu 51 on a plurality of 
pages, navigation information such as a scroll bar or 

20 icons for displaying the next page or previous page is 
produced independently by the controller and displayed 
on the display screen. 

[0345] Data objects (still pictures and text data) 
showing the configuration set parts 55 themselves are 

25 used as captions on the display screen of the controller. 
[0346] The configuration set parts contained infor- 
mation about priority in this embodiment, but the identi- 
fiers (such as ID) of the configuration set parts may also 
be used, and a lower value of an identifier can be 

30 treated as a higher priority, allowing the same effect to 
be obtained. 

[0347] When the user clicks on the icon indicating 
"next page" in the operation screen in Figure 10B, the 
controller detects this operation with the display/func- 
35 tion selection means 14, and displays on the screen the 
configuration set part (VTR tuner) indicating the next 
priority, for example. Text data and an icon indicating the 
"previous menu" are displayed here independently by 
the controller. 

40 [0348] While not depicted, the display parts 52 not 
belonging to the configuration set parts are displayed on 
a page after all the configuration set parts. 
[0349] Out of "rewind," "play," "fast forward," 
"pause," "stop," and "record" belonging to the configura- 

45 tion set parts 55 shown in Figure 10B, the functions 
other than "stop," which does not handle stream data, 
use channel information (CH 5). 
[0350] In Figure 10C, the display parts 52 indicated 
by "channel up" and "channel down" use CH 6. 

so [0351] Here, when the controller displays stream 
data, it may be displayed along with the configuration 
set parts 55 that produce the device, or only the stream 
data may be displayed and the configuration set parts 
55 displayed once again by operation of the remote con- 

55 trol 22 on the part of the user. 

[0352] The configuration set parts 55 can also be 
used hierarchically, and there may be separate configu- 
ration set parts 55 among the configuration set parts 55. 
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In this case it is possible to partition th display parts 52 
mor finely, allowing for even more flexibility in the 
screen size of th controller. 

[0353] Also, as shown in Figur 19, the menus 
(function menus) that show the functions of the device 5 
may contain channel information, in which case, of the 
display parts 52 belonging to the function menu 51, 
channel information (CH 8) is used for those functions 
that transmit stream data using a channel on the trans- 
mission line 1 . Thus, the flow of stream data can be con- 
trolled easily in device units, and the user can be easily 
provided with functions that bridge devices. 
[0354] As discussed above, with the present inven- 
tion, a device has a plurality of display parts that make 
up the operation screen of this device, and configuration 
set parts that indicate a set consisting of a number of 
the above-mentioned display parts and that contain 
channel information indicating the channel of the trans- 
mission line, and a controller reads these display parts 
and configuration set parts from this device and sets the 
channel of the transmission line according to the chan- 
nel information in the handling of data shown by a dis- 
play part belonging to a configuration set part, and as a 
result, the channel of the transmission line can be set in 
the various function units among the devices shown by 
the configuration set parts, it is possible to control the 
flow of stream data in the various function units with 
respect to a device having a plurality of function units, 
and simultaneous use in function units is also possible. 
Also, if the display parts are displayed on the display 
screen according to the information of a configuration 
set part having channel information, then even when 
only a display screen that is smaller than the screen size 
the device is supposed to have can be used with the 
controller, and the menu of the device cannot be dis- 
played completely due to limitations such as the resolu- 
tion of the screen or the effect of other display 
categories, an operation screen that is easy for the user 
to understand can still be provided by dividing the dis- 
play into a plurality of pages according to the configura- 
tion set part information, and stream data can be 
handled independently for each individual page. 
[0355] If the device has menus which consist of a 
plurality of display parts showing the operation screen 
of the device and which contain channel information 
showing the channels of the transmission line, and if the 
controller reads the menus and display parts from the 
device and sets the channel of the transmission line 
according to the channel information in the handling of 
the data shown by the above-mentioned display parts 
belonging to the menus, then the channels of the trans- 
mission line can be easily set in device units, and it will 
be possible to handle stream data with ease. 
[0356] If configuration set parts or menus have 
flags indicating the type of stream, then stream data that 
cannot be handled by the controller can be detected 
befor the controller displays, and it is possible to notify 
the user that the function shown by this display part 52 



cannot be used with this controller, which means that an 
op ration scr en which is easi r to use can be pro- 
vided. 

Eighth Embodiment: Content display 

[0357] The eighth embodiment will be described 
using Figures 20 to 23. Here, content flags were pro- 
vided for display parts such as icons whose attributes 
included video (content) that was displayed upon 
receipt of a broadcast or the like, and only the display 
parts including content could be gathered. 
[0358] The eighth embodiment of the present 
invention will now be described through reference to the 
figures. 

[0359] Figure 20 is a structural diagram illustrating 
the function menu and configuration set parts in the 
eighth embodiment. Everything but the content display 
parts here is the same as in the first embodiment, and 
therefore will not be described. 

[0360] In Figure 20, the various content display 
parts 62 in these function information tables 8 (not 
shown) have channel information on the transmission 
line 1. Furthermore, the various content display parts 62 
have content flags showing information about the con- 
tent in the device. There may be flags in the header 
showing that these are the content display parts 62, or 
these may be distinguished from other display parts and 
so forth by identifiers (ID's) or the like. The other constit- 
uent elements are the same as with the display parts 
52. For example, as shown in Figure 20, a still picture 
object having an entry in the content display parts 62 of 
this device (VTR) was produced from a single still pic- 
ture extracted from the playback picture of the device. 
[0361] First, when this device is connected to the 
transmission line 1 , the controller on this transmission 
line 1 recognizes a new device by bus resetting, in the 
case of a 1394 bus, for instance, and reads the device 
information parts 50 of the function information tables 8 
through the transmission line 1 from the new device as 
directed by the function information table management 
means 12, so that these device information parts 50 of 
the function information tables 8 are read and registered 
in the function data base 13 of the controller. Next, the 
function menu 51 and the content display parts 62 are 
read by the same method as that discussed in the first 
embodiment. 

[0362] The function information tables 8 here are 
the ones shown in Figure 14, and the function menu 51 
and the content display parts 62 are the ones shown in 
Figure 20. These content display parts 62 have data 
objects showing the various functions (in Figure 20, text 
data and still picture data for "marathon race" and "gym- 
nastics competition"), control codes (in Figure 20, the 
ID's of the various content display parts), and so forth. 
Content display parts using a channel on the transmis- 
sion line 1, such as the content display parts 62, have 
channel information sent by the device as stream data 
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when this function is selected. For instance, a channel 
performs broadcast out with IEEE 1394 AV protocol. 
This channel information also determines which chan- 
nel the unit internal control means 9 in the device will 
reserve for sending stream data, and is given in the var- 5 
ious content display parts 62 of the function information 
tables 8 in the device. (The "marathon race" is on chan- 
nel 2, and the "gymnastics competition on channel 3.) 
[0363] The various parts in the function information 
tables 8 do not necessarily need to physically contain 
the links shown in Figure 14 (association of entries and 
entities), as long as they can be accessed from the con- 
troller in object units by identifiers (ID's) or the like. 
[0364] The display/function selection means 14 
refers to the function information tables 8 (or some of 
the function information tables 8) in the function data 
base 13 through the function information table manage- 
ment means 12. 

[0365] If the display/function selection means 14 
displays a list of the devices connected to this controller, 
then the display/function selection means 14 uses the 
function information table management means 12 to 
read the data objects (such as text objects and still pic- 
ture objects) in the device information parts 50 from the 
function information tables 8 of all of the devices regis- 
tered in the function data base 13, and displays these 
are on the screen. 

[0366] Next, when the user selects a still picture 
object of a device by means of the pointing function 
(such as the directional key) of a remote control, the dis- 
play/function selection means 14 uses the function 
information table management means 12 to read the 
content display parts 62 of the various functions given in 
the function menu list from the function information table 
8 of this device, and displays the content display parts 
62 corresponding to the various functions on the 
screen. This makes it possible to display on the screen 
the content display parts 62 showing all of the functions 
of this device. Here, the function menu 51 shown in Fig- 
ure 20 is displayed as in Figure 21, for example. 
[0367] Next, when the user selects a content dis- 
play part 62 showing, for example, the play function of 
the device (the still picture showing "marathon race") by 
means of the pointing function of a remote control or the 
like, the display/function selection means 14 sends the 
ID of this content display part 62 attached by the device 
as a control code to the device. 

[0368] The configuration may also be such that the 
operation information of the user is sent to the device 
along with the control code, as in embodiment 1. 
[0369] The device receives the control code (ID) of 
the content display part 62 selected on the controller, 
the meaning of the received data is interpreted by the 
device asynchronous data processing means 6, it is 
determined that the "marathon race" of the VTR is to be 
played, and a play directive is issued to the unit internal 
control means 9. At this point the unit internal control 
means 9 reserves the band of the transmission line 1 



required for VTR playback, and acquires a channel as a 
predetermined value. The unit internal control means 9 
searches for the program titled "marathon race" 
recorded by the VTR, goes to the beginning of the pro- 
gram, and starts playing, and stream data is outputted 
to a channel (CH 2) on the transmission line 1. 
[0370] The controller waits for a response from the 
device with respect to the control code sent to the 
device, and after the proper response has been 
returned, sets the channel of the transmission line 1 
given in the channel information (CH 2) to receive, 
reads the packet flowing through channel 2 of the trans- 
mission line 1 , and performs processing such as decod- 
ing with the controller signal processing means 10, after 
which the display/function selection means 14 displays 
the image on the display screen. 
[0371] In this case, if the device wants to change a 
content display part 62, such as inverting the color of 
the still picture of the "marathon race," for example, in 
order to show that the "marathon race" is being played, 
then the device sends the content display part 62 to be 
changed (including text objects and still picture objects) 
to the controller. Thus, the controller learns that the con- 
tent display part 62 of this ID has been changed, and 
overwrites the corresponding content display part 62. 
[0372] Here, if the device wants to change a content 
display part, the ID of the content display part 62 may be 
sent to the controller, in which case the controller uses 
this ID to retrieve this content display part 62 from the 
device, and overwrites the corresponding content dis- 
play part 62. The ID, or the ID and data, may also be 
sent to the controller in data object units rather than 
content display part units. 

[0373] If, for some reason, the device is unable to 
receive a control code sent by the controller, the dis- 
play/function selection means 14 detects that the device 
did not receive the control code based on its response 
to the transmitted control code or the like, and a content 
display part 62 independently had by the controller is 
displayed (displayed in a large size in a conspicuous 
location such as the center of the screen, for example), 
thereby notifying the user that control failed. 
[0374] When content playback is to be ended, the 
controller sends the device the control code (ID) of the 
content display part 62 showing the content along with 
information showing the end of playback. The device 
receives the control code and this information showing 
the end of playback, ends the playback of the content, 
and releases the channel and band of the transmission 
line 1 that had been reserved. 

[0375] The controller received the stream flowing 
through the corresponding channel of the transmission 
line 1 in this embodiment, but it is also possible for the 
controller to direct another device or controller to 
receive the stream data flowing through the correspond- 
ing channel, which makes it easy to direct that the 
stream data flowing through th corresponding channel 
be received, allows the controller to control the flow of 
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the stream easily and freely in function units, and allows 
more functions to be provided to the user. 
[0376] Also, the stream data flowing through the 
corresponding channel of th transmission line 1 was 
video/ audio data here, but may instead be just video or 5 
just audio data, or other data (character data or print 
data used with a printer), and in more specific tens, may 
be MPEG or DV data, and as long as the function shown 
by a display part 52 transfers data over the transmission 
line 1 , any such data may be used as stream data. 
[0377] Furthermore, if the display parts 52 have 
flags indicating the type of this stream data, stream data 
that cannot be handled by the controller can be 
detected before the controller displays this display part 
on the screen, and it is possible to notify the user that 
the function shown by this display part 52 cannot be 
used with this controller, which means that an operation 
screen which is easier to use can be provided. 
[0378] The device determined the channel informa- 
tion ahead of time in this embodiment, but after the 
device has received a control code from the controller, a 
band and channel may be reserved on the transmission 
line 1 , and this channel information sent to the control- 
ler. For example, the configuration may be such that 
when the user presses a still picture given as a content 
display part 62 having channel information showing that 
one of the channels on the transmission line 1 is being 
used (that is, information showing that a channel has 
been determined after selection of this unit), the device 
receives the control code of this still picture, and sends 
the controller a content display part 62 including chan- 
nel information acquired by the device after receipt of a 
control code and a still picture showing that this play 
button has been pressed (such as one in which the 
color of the still picture has been inverted). 
[0379] Another possibility is for channel information 
scheduled for use by the device to be provided to the 
content display part 62, and after the control code of this 
content display part 62 has been received by the device 
from the controller, the device reserves the band and 
channel, and the specified channel information is sent 
to the controller as a response to the control code. 
[0380] The configuration may also be such that the 
channel information is rewritable, the device reserves a 
band and channel when another controller is used the 
device, for example, the channel information of the cor- 
responding content display part 62 in the function infor- 
mation table 8 is updated as needed, and the controller 
is notified that this content display part has been 
changed. In this case, a change in the status of the 
device by another controller (such as the acquired chan- 
nel of the device) can be easily accommodated by dis- 
playing the output stream of the device. 
[0381] With a disk apparatus such as a DVD, if 
every content display part 62 has channel information, 
then the controller will be able to execute and direct a 
plurality of functions simultaneously, and it will be possi- 
ble to display a plurality of streams simultaneously on 



the screen of the controller (watching two contents at 
th same time), to watch on program while recording a 
background program, and so on. Also, when another 
controller is controlling the device in question, it is pos- 
sible for th controller to receive and display on the 
screen the channel on the transmission line 1 given in 
the channel information, allowing the user to be notified 
of the status of a unit being used by another controller 
through video/audio played on the display screen or the 
like, and allowing an operation screen that is easier for 
the user to understand to be constructed. 
[0382] The still pictures showing the content in the 
device were produced from playback images of the 
device in this embodiment, but with an STB or the like, it 
is also possible for the configuration to be such that the 
still pictures showing the content from the broadcast are 
retrieved from information such as EPG, or, with a 
recording device such as a VTR or a DVD-RAM, to pro- 
duce still pictures showing the content during recording 
and store them as index information in a storage 
medium or the device, or, in the case of a non-recorda- 
ble device such as a DVD-ROM, to store these still pic- 
tures as index information in a storage medium ahead 
of time. 

[0383] Only the functions showing the content in the 
device were described in this embodiment, but it is also 
possible to provide flags showing content information in 
the header of the content display parts 62 for functions 
showing content, which distinguishes these functions 
from operation functions of the device such as "play," 
and to have both types of function present in a single 
function menu 51 or configuration set part 55, thereby 
making the various functions more understandable to 
the user. 

[0384] Furthermore, the control codes (ID's) of the 
content display parts 62 showing the content were used 
in ending the playback of content in this embodiment, 
but the content display parts may have separate control 
codes indicating the end of content playback. Also, the 
ending of content playback and the release of the band 
and channel of the transmission line 1 may be divided, 
and control codes indicating each may be added to the 
content display part 62 apart from the ID's. The configu- 
ration may also be such that the control codes (ID's) of 
the content display parts 62 showing the content and 
information showing the end of playback, or the control 
codes (ID's) and information showing the release of the 
band and channel, are transmitted to the device. 
[0385] Also, as shown in Figure 22, the channel 
information may be contained by content configuration 
set parts 65 showing sets of display parts 52 that do not 
have content flags but show the contents of the devices, 
in which case, channel information (CH 5) is used for 
the functions transmitting stream data ("marathon race" 
and "gymnastics competition") using a channel on the 
transmission line 1 out of the display parts 52 belonging 
to the content configuration set part 65. 
[0386] Here, the content configuration set part 65 
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has content flags showing that display parts 52 showing 
content information are included in this content configu- 
ration set part 65. Differentiation from other display 
parts and the like may b achieved with identifi rs(ID*s) 
or the like, or by having flags indicating th cont nt con- 
figuration set part 65 in the header. 
[0387] The display parts 52 here do not have flags 
showing content information, but make the user recog- 
nize that content information is being shown, by data 
objects such as still pictures. 

[0388] As to the reading of the content configuration 
set parts 65, it is the same as for the configuration set 
parts 55 in the seventh embodiment, and will therefore 
not be described. 

[0389] The configuration set parts 55 have informa- 
tion showing the end of playback of the content of the 
display parts 52, and when the content playback of the 
display parts 52 belonging to the configuration set parts 
55 is ended, the information showing this ending of con- 
tent playback is sent to the device. 
[0390] Thus, it is possible for the device to specify 
the channel of the transmission line 1 in the function 
units of the device, allowing suitable channel informa- 
tion to be shared by the device and controller with a 
smaller quantity of information, and allowing the control- 
ler to control the flow of a stream easily and freely in 
function units. 

[0391] The ending of content playback and the 
release of the band and channel of the transmission line 
1 may be divided, and information indicating each may 
be added as control codes to the configuration set parts 
55 apart from the ID's. 

[0392] There may be content display parts 62 in the 
configuration set parts 55, in which case the channel 
number of the content display parts 62 will take priority 
over these content display parts 62 when the content 
display parts 62 have a channel number. 
[0393] As shown in Figure 23, a menu showing the 
content of a device (content menu) may contain channel 
information, in which case channel information (CH 8) is 
used for the functions transmitting stream data using a 
channel on the transmission line 1 out of the display 
parts 52 belonging to the content menu 61. Here, the 
display parts 52 belonging to the content menu 61 have 
no flags indicating that they are content information, but 
show the content in the device. Furthermore, these dis- 
play parts 52 make the user recognize that content 
information is being shown, by data objects such as still 
pictures. 

[0394] The content menu 61 here has content flags 
showing that display parts 52 showing content informa- 
tion are included in this content menu 61 . Differentiation 
from other display parts and the like may be achieved 
with identifiers (ID's) or the like, or by having flags indi- 
cating the content menu 61 in the header. 
[0395] The content menu 61 has information show- 
ing the end of playback of the content of the content dis- 
play parts 62, and when the content playback of the 



display parts 52 belonging to the content menu 61 is 
ended, th information showing this ending of content 
playback is sent to th device. 

[0396] Thus, it is possible for the device to specify 
5 the channel of the transmission line 1 in the function 

units of the device, allowing enough required channel 

information to be shared by the device and controller. 

[0397] The ending of content playback and the 

release of the band and channel of the transmission line 
10 1 may be divided, and information indicating each may 

be added as control codes to the content menu 61 apart 

from the ID's. 

[0398] Furthermore, there may be content display 
parts 62 in the content menu 61, in which case the 

15 channel number of the content display parts 62 will take 
priority over these content display parts 62 when the 
content display parts 62 have a channel number. 
[0399] As discussed above, with this embodiment, 
the device has content display parts that make up the 

20 operation screen of the device, show the content han- 
dled by the device, and contain control codes specifying 
content, while the controller reads the content display 
* parts from the device and displays the content display 
parts on the display screen, so when the user operates 

25 a content display part on the display screen, the control- 
ler uses the control code to direct the transmission and 
receipt of the content to and from this device, which 
makes it possible to notify the user of the content of the 
device and allows an operating environment that is eas- 

30 ier to understand to be provided. 

[0400] If the content display parts have channel 
information showing the channel of the transmission 
line over which the contents showing the content display 
parts are transmitted and received, then it will be possi- 

35 ble for the controller to handle with ease the stream data 
of content shown by the content display parts, it will be 
possible to display the content on the screen of the con- 
troller with a simple configuration, and it will be possible 
to direct the transmission of stream data to other 

40 devices. It will further be possible for a plurality of con- 
tents to be handled simultaneously according to the 
capability of the device. 

[0401] The device has content display parts that 
make up the operation screen of the device, show the 

45 content handled by the device, and contain control 
codes specifying content, and content configuration set 
parts that show a set composed of a plurality of content 
display parts, while the controller reads the content dis- 
play parts and content configuration set parts from the 

50 device and displays the content display parts on the dis- 
play screen according to the information shown by the 
content configuration set parts, so when the user oper- 
ates a content display part on the display screen, the 
controller uses the control code to direct the transmis- 

55 sion and receipt of the content to and from this device, 
which allows for the simple classification of contents for 
which the device is able to send stream data simultane- 
ously, and the user can be easily made to recognize 
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contents which cannot b s lected simultaneously. 
[0402] The device has content display parts that 
make up the operation screen of the device, show the 
content handled by the device, and contain control 
codes specifying content, and a content menu that is 5 
composed of one or more content display parts and 
shows the functions of the device, while the controller 
reads the content menu and the content display parts 
from the device and displays the content menu on the 
display screen, so when the user operates a content 
display part on the display screen, the controller uses 
the control code to direct the transmission and receipt of 
the content to and from this device, which allows the 
contents handled by the device to be displayed simply 
and clearly on the operation screen of the controller, 
and makes it possible for the contents shown by these 
content display parts to be played back with ease, 
merely by selecting a content display part. 
[0403] The content display parts, the content con- 
figuration set parts, or the content menus have flags 
showing the type of stream, which makes it possible for 
stream data that cannot be handled by the controller to 
be detected before the controller displays anything on 
the screen, and it is possible to notify the user that the 
function shown by this display part 52 cannot be used 
with this controller, which means that an operation 
screen which is easier to use can be provided. 

Effect of the Invention 

[0404] A device has a plurality of display parts that 
constitute the operation screen of the device, these dis- 
play parts contain layout information indicating the lay- 
out relationship to be assumed with other display parts, 
the controller reads the display parts from the device, 
and the display parts are displayed on the display 
screen according to the layout information, which 
means that even when only a display screen that is 
smaller than the screen size the device is supposed to 
have can be used with the controller, and the menu of 
the device cannot be displayed completely due to limita- 
tions such as the resolution of the screen or the effect of 
other display categories, an operation screen that is 
easy for the user to understand can still be provided by 
rearranging the display parts according to the layout 
information or by dividing the display into a plurality of 
pages. 

[0405] If the layout information is relationship infor- 
mation having the same information for a plurality of dis- 
play parts to be laid out in proximity, then even when a 
plurality of display groups are present in a single menu, 
or when there are numerous display parts belonging to 
a display group to be laid out in proximity, and even 
when a plurality of display parts make up a plurality of 
groups, the layout information can be described simply 
with appended information of small data quantity, and 
even when only a display screen that is smaller than the 
screen size the device is supposed to have can be used 



with the controller, and the menu of the devic cannot 
be displayed completely due to limitations such as the 
resolution of the screen or th effect of other display cat- 
gories, an operation screen that is easy for the user to 
understand can still be provided by rearranging the dis- 
play parts according to the layout information or by 
dividing the display into a plurality of pages. 
[0406] if a single display part has a plurality of sets 
of layout information, then the layout information can be 
described simply even when there are a plurality of dis- 
play parts to be laid out in proximity with respect to a 
single display part. Even when only a display screen 
that is smaller than the screen size the device is sup- 
posed to have can be used with the controller, and the 
menu of the device cannot be displayed completely due 
to limitations such as the resolution of the screen or the 
effect of other display categories, closely related display 
parts can still be displayed in proximity and always 
within a single screen and an operation screen that is 
easy for the user to understand and operate can still be 
provided by displaying a single display part a plurality of 
times in the division and display of the display parts on 
a plurality of pages according to the layout information. 
[0407] If the layout information is identifiers of dis- 
play parts to be laid out in proximity, then it is easier to 
define the layout information, the controller can directly 
detect display parts to be laid out in proximity, it is pos- 
sible for display parts that are to be laid out in proximity 
to be displayed on the display screen more simply, and 
even when only a display screen that is smaller than the 
screen size the device is supposed to have can be used 
with the controller, and the menu of the device cannot 
be displayed completely due to limitations such as the 
resolution of the screen or the effect of other display cat- 
egories, an operation screen that is easy for the user to 
understand can still be provided by rearranging the dis- 
play parts according to the layout information or by 
dividing the display into a plurality of pages. 
[0408] If the display parts have identifiers of display 
parts to be laid out in proximity in the up, down, left, and 
right directions, then the device can specify the display 
method more precisely, and the intent of the person who 
created the device menu can be conveyed easily to the 
user even when the controller only has a display screen 
that is smaller than the screen size the device is sup- 
posed to have. 

[0409] If a device has a plurality of display parts that 
make up the operation screen of this device, and config- 
uration set parts that indicate a set consisting of a 
number of the display parts and that have information 
indicating the priority of the display, and if a controller 
reads these display parts and configuration set parts 
from this device and displays the display parts on a dis- 
play screen according to the information of the configu- 
ration set parts, then even when only a display screen 
that is smaller than the screen size the device is sup- 
posed to have can be used with th controller, and the 
menu of the device cannot be displayed completely due 
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to limitations such as the resolution of the scr en or the 
effect of other display categories, it is possible for the 
controller to display the information on the scr en in the 
order of its importance to the user, the result of which is 
a display screen that is easier to view. Furthermore, the 5 
menu producer is able to create an optimal design for a 
given screen size more freely, and is also able to include 
in the design such considerations as how the individual 
screens appear for different screen sizes, and particu- 
larly the division of the pages and the relationship w 
between the preceding and succeeding pages. 
[041 0] If a device has a plurality of display parts that 
make up the operation screen of this device, and config- 
uration set parts that indicate a set consisting of a 
number of the display parts, and if a controller reads the is 
display parts and set information from the device and 
displays the display parts on the display screen accord- 
ing to the information of the configuration set parts, then 
even when only a display screen that is smaller than the 
screen size the device is supposed to have can be used 20 
with the controller, and the menu of the device cannot 
be displayed completely, an operation screen that is 
easy for the user to understand can still be provided by 
dividing the display into a plurality of pages according to 
the information of the configuration set parts. 25 
[0411] Furthermore, the producer of the device 
menu can assume a plurality of screen sizes in produc- 
ing the menu, and the intent of the menu producer can 
be conveyed to the user even for controllers of different 
screen sizes. Also, the controller is able to handle the 30 
display parts and display them on the screen in configu- 
ration set part units, which simplifies processing. 
[0412] If the configuration set parts have a hierar- 
chical structure, then the menu producer will be able to 
assume more screen sizes in the production of the 35 
device menus, and the controller can display variously 
suitable menu screens for more screen sizes. 
[0413] If the device has a menu consisting of a plu- 
rality of display parts showing the operation screens of 
this device, the controller reads these display parts from 40 
this device, and when the size of this menu is larger 
than the display screen of the controller, the controller 
divides the display into a plurality of pages the same 
size or smaller than the display screen, and produces 
information about navigation between these plurality of 45 
pages and displays this information on the display 
screen, then even when only a display screen that is 
smaller than the screen size the device is supposed to 
have can be used with the controller, and the menu of 
the device cannot be displayed completely due to limita- so 
tions such as the resolution of the screen or the effect of 
other display categories, an operation screen that is 
easy for the user to understand can still be provided by 
dividing the display into a plurality of pages. Further- 
more, controllers of a variety of display screen sizes can ss 
be easily accommodated by virtue of the controller hav- 
ing the initiative in regard to the production of informa- 
tion about navigation between these plurality of pages. 



[0414] If device has display parts having operation 
information for changing the display of the operation 
screen by operation from th user, and a menu consist- 
ing of a plurality of display parts showing the op ration 
screen, and if the controller reads these display parts 
and configuration set parts from this device and dis- 
plays the menu of the device on the display screen, and 
the display of the display screen is changed by opera- 
tion from the user according to the operation informa- 
tion, then the menu producer can assume operation by 
the user in the production of the menu, so an operation 
screen that is easy to understand can be provided to the 
user, and since there is no need to read the information 
of the display screen from the device every time the 
user performs an operation, the display screen can be 
changed more quickly. Furthermore, when only a dis- 
play screen that is smaller than the screen size the 
device is supposed to have can be used with the con- 
troller, even if the positions of the various display parts 
have been rearranged by the controller, the user can be 
simply and clearly apprised of the provision of the vari- 
ous functions and menus by defining the movement of 
the cursor with operation information, so an operation 
screen that is easy to operate can be provided. 
[0415] Furthermore, using identifiers showing other 
display parts for the operation information makes it sim- 
ple to define the movement of the cursor, and allows the 
provision of the various functions shown by the display 
parts to be presented in a straightforward manner to the 
user, so the user is able to learn of the relationship 
between the functions intuitively, and since there is no 
need to ask the device about cursor movement, opera- 
tion is easier. 

[0416] Also, if the menus have configuration set 
parts, and the operation information is identifiers indi- 
cating configuration set parts, then the user can be eas- 
ily apprised of the association between the various 
configuration set parts, allowing the provision of an 
operation screen that is easier to use. 
[0417] If the device has a plurality of menus, and if 
display parts having as operation information identifiers 
indicating other menus are provided within the menus, 
then menus can be switched with ease, even though 
there is no button for menu switching or the like on the 
remote control, etc. 

[0418] Furthermore, if the device has a help menu 
giving explanations of the functions shown by the vari- 
ous devices, and if the display parts have as operation 
information identifiers indicating the help menu, then an 
operation screen that can display explanations of the 
various functions and menus with ease and that is eas- 
ier to use can be provided to the user. 

Effect of the Invention 

[0419] If a device has display parts which make up 
the operation screen of the device and contain channel . 
information that shows the channel of the transmission 
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lin , and if the controller r ads the display parts from the 
device and sets the chann I of the transmission line 
according to the chann I information when handling 
data shown by the display parts, then it is possible to 
share channel information between the controller and 5 
the device with a simple configuration, allowing the con- 
troller to control the flow of stream data with ease. Also, 
a plurality of data streams can be executed simultane- 
ously if there is channel information for every display 
part with a device that can handle a plurality of data w 
streams simultaneously. 

[0420] If a device has a plurality of display parts that 
make up the operation screen of this device, and config- 
uration set parts that indicate a set consisting of a 
number of the display parts and that contain channel 15 
information indicating the channel of the transmission 
line, and if a controller reads these display parts and 
configuration set parts from this device and sets the 
channel of the transmission line according to the chan- 
nel information in the handling of data shown by a dis- 20 
play part belonging to a configuration set part, then the 
channel of the transmission line can be set in the vari- 
ous function units among the devices shown by the con- 
figuration set parts, it will be possible to control the flow 
of stream data in the various function units with respect 25 
to a device having a plurality of function units, and 
simultaneous use in function units will also be possible. 
Also, if the display parts are displayed on the display 
screen according to the information of a configuration 
set part having channel information, then even when 30 
only a display screen that is smaller than the screen size 
the device is supposed to have can be used with the 
controller, and the menu of the device cannot be dis- 
played completely due to limitations such as the resolu- 
tion of the screen or the effect of other display 35 
categories, an operation screen that is easy for the user 
to understand can still be provided by dividing the dis- 
play into a plurality of pages according to the configura- 
tion set part information, and stream data can be 
handled independently for each individual page. 40 
[0421] If the device has menus which consist of a 
plurality of display parts showing the operation screen 
of the device and which contain channel information 
showing the channels of the transmission line, and if the 
controller reads the menus and display parts from the 45 
device and sets the channel of the transmission line 
according to the channel information in the handling of 
the data shown by the display parts belonging to the 
menus, then the channels of the transmission line can 
be easily set in device units, and it will be possible to so 
handle stream data with ease. 

[0422] If display parts, configuration set parts, and 
menus have flags indicating the type of stream, then 
stream data that cannot be handled by the controller 
can be detected before the controller displays, and it is 55 
possible to notify the user that the function shown by 
this display part cannot be used with this controller, 
which means that an operation screen which is easier to 



use can be provided. 

[0423] If a device has content display parts that 
make up th operation screen of the device, show the 
content handled by the device, and contain control 
codes specifying content, and if the controller reads the 
content display parts from the device and displays the 
content display parts on the display screen, so that 
when the user operates a content display part on the 
display screen, the controller uses the control code to 
direct the transmission and receipt of the content to and 
from this device, then it will be possible to notify the user 
of the content of the device and will allow an operating 
environment that is easier to understand to be provided. 
[0424] If a device has content display parts that 
make up the operation screen of the device, show the 
content handled by the device, and contain control 
codes specifying content, and if the controller reads the 
content display parts and from the device and displays 
the content display parts on the display screen, so that 
when the user operates a content display part on the 
display screen, the controller uses the control code to 
direct the transmission and receipt of the content to and 
from this device, then it will be possible to notify the user 
of the contents of the device, and an operating environ- 
ment that is easier to understand can be provided. 
[0425] If the content display parts have channel 
information showing the channel of the transmission 
line over which the contents showing the content display 
parts are transmitted and received, then it will be possi- 
ble for the controller to handle with ease the stream data 
of content shown by the content display parts, it will be 
possible to display the content on the screen of the con- 
troller with a simple configuration, and it will be possible 
to direct the transmission of stream data to other 
devices. It will further be possible for a plurality of con- 
tents to be handled simultaneously according to the 
capability of the device. 

[0426] If a device has content display parts that 
make up the operation screen of the device, show the 
content handled by the device, and contain control 
codes specifying content, and content configuration set 
parts that show a set composed of a plurality of content 
display parts, and if the controller reads the content dis- 
play parts and content configuration set parts from the 
device and displays the content display parts on the dis- 
play screen according to the information shown by the 
content configuration set parts, so that when the user 
operates a content display part on the display screen, 
the controller uses the control code to direct the trans- 
mission and receipt of the content to and from this 
device, then the contents for which the device is able to 
send stream data simultaneously can be classified with 
ease, and the user can be easily made to recognize 
contents which cannot be selected simultaneously. 
[0427] If a device has content display parts that 
make up the operation screen of the device, show the 
content handled by the device, and contain control 
codes specifying content, and a content menu that is 
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composed of one or more content display parts and 
shows the functions of the device, and if the controller 
reads the content menu and the content display parts 
from the device and displays the content menu on the 
display screen, so that when the user operates a con- 
tent display part on the display screen, the controller 
uses the control code to direct the transmission and 
receipt of the content to and from this device, then the 
contents handled by the device can be displayed simply 
arid clearly on the operation screen of the controller, 
and it will be possible for the contents shown by these 
content display parts to be played back with ease, 
merely by selecting a content display part. 
[0428] If the content display parts, the content con- 
figuration set parts, or the content menus have flags 
showing the type of stream, then it will be possible for 
stream data that cannot be handled by the controller to 
be detected before the controller displays anything on 
the screen, and it will be possible to notify the user that 
the function shown by this display part cannot be used 
with this controller, which means that an operation 
screen which is easier to use can be provided. 
[0429] If a device has display parts that make up 
the operation screen of the device, and menus com- 
posed of one or more display parts and showing the 
functions of the device, and if, when the controller 
requests a usage right for a device and the device 
grants this usage right, the device sends the controller a 
menu list including identifiers for the display parts that 
make up the above-mentioned menus of the device, 
then the controller can acquire the menus in the device 
by a simple procedure, the number of display parts in a 
menu list and so on can be confirmed even when there 
is little memory space in the controller, it will be possible 
to read just what is needed, and the menus of the 
device can be displayed more efficiently. 
[0430] When there is a change in the display parts 
in a device after the controller has requested the usage 
right for that device, the device can easily ascertain the 
controller having the usage right by sending the above- 
mentioned changed display parts to the controller, mak- 
ing it possible for a notification of a change in the display 
parts in a device to be made efficiently when there is 
such a change. When the controller relinquishes its 
usage right to the device, the controller notifies the 
device that the usage right has been relinquished, and 
the channel of the transmission line that was in use at 
the directive of this controller is released by the device, 
which prevents the device from remaining in a state in 
which the channel of the transmission line is in use, and 
allows the transmission line to be utilized more effec- 
tively. 

Claims 

1 . An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 



nected via a transmission lin , of these units, the 
one having a user interface is t rmed a controller, 
and the units that are controlled are termed 
devices, 

5 

said devices having means for holding a plural- 
ity of display parts that constitute an operation 
screen for said devices, 
at least one of the display parts having layout 

10 information showing the layout relationship to 

be assumed with the other display parts, 
said controller having means for reading said 
display parts from said devices, and 
said display parts being displayed on a display 

15 screen according to said layout information. 

2. The network control system according to Claim 1 , 
wherein the layout information is relationship infor- 
mation having the same information for a plurality of 

20 display parts to be laid out in proximity. 

3. The network control system according to Claim 1 , 
wherein the layout information comprises identifiers 
of the display parts to be laid out in proximity. 

25 

4. The network control system according to claim 3, 
wherein the display parts have identifiers for the 
display parts to be laid out in proximity in the direc- 
tions of up, down, left, and right. 

30 

5. The network control system according to claim 2 or 
3, wherein one display part has a plurality of sets of 
layout information. 

35 6. A device in an audio/video network control system 
in which two or more units that handle one or more 
sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 

40 controller, and the units that are controlled are 
termed devices, 

said devices having means for holding a plural- 
ity of display parts that constitute an operation 
45 screen, and 

at least one display part having layout informa- 
tion showing the layout relationship to be 
assumed with the other display parts. 

so 7. A controller in an audio/video network control sys- 
tem in which two or mare units that handle one or 
more sets of data pertaining to video, audio, or 
information are connected via a transmission line, 
of these units, the one having a user interface is 

55 termed a controller, and the units that are controlled 
are termed devices, 

said controller having means for reading from 
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the devices th display parts that constitute an 
operation screen for said devices and layout 
information showing the layout relationship that 
said display parts should hav with the other 
display parts, 5 
and said display parts being displayed on a dis- 
play screen according to said layout informa- 
tion. 

8. An audio/video network control system in which two w 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed is 
devices, 

said devices having means for holding a plural- 
ity of display parts that constitute an operation 
screen for said devices, 20 
a set consisting of a number of display parts 
being shown from among said display parts, 
there being configuration set parts having infor- 
mation showing the priority order of display, 
said controller reading said display parts and 25 
said set information from said devices, and 
said display parts being displayed on a display 
screen according to information about said 
configuration set parts. 

30 

9. The network control system according to Claim 8, 
wherein the controller displays each page of an 
operation screen in units of information about each 
configuration set part. 

35 

10. The network control system according to Claim 8, 
wherein the configuration set parts have a hierar- 
chical structure. 

11. A device in an audio/video network control system 40 
in which two or more units that handle one or more 
sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 
controller, and the units that are controlled are 45 
termed devices, 

said devices having: 

a plurality of display parts that constitute an 
operation screen for said devices, and so 
configuration set parts showing a set consist- 
ing of a number of display parts from among 
said display parts. 

12. A controller in an audio/video network control sys- 55 
tern in which two or more units that handle one or 
more sets of data pertaining to video, audio, or 
information are connected via a transmission line, 



of these units, the on having a user interfac is 
termed a controller, and th units that are controlled 
are termed devices, 

said controller having means for reading a plu- 
rality of display parts that constitute an opera- 
tion screen for said devices, and configuration 
set parts showing a set consisting of a number 
of display parts from among said display parts, 
and 

said display parts being displayed on a display 
screen according to information about said 
configuration set parts. 

13. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 

said devices having a menu consisting of a plu- 
rality of display parts showing an operation 
screen for said devices, 
said controller having means for reading said 
display parts from said devices, and 
when said menu is larger than the display 
screen of said controller, the display of said 
controller is divided into a plurality of pages the 
same size as or smaller than said display 
screen, and said controller produces informa- 
tion about navigation between said plurality of 
pages and displays this information on said dis- 
play screen. 

14. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 
more sets of data pertaining to video, audio, or 
information are connected via a transmission line, 
of these units, the one having a user interface is 
termed a controller, and the units that are controlled 
are termed devices, 

said controller having means for reading 
menus consisting of a plurality of display parts 
showing an operation screen for said devices 
. from the devices being controlled, and 
when said menu is larger than the display 
screen of said controller, the display of said 
controller being divided into a plurality of pages 
the same size as or smaller than said display 
screen, and said controller producing informa- 
tion about navigation between said plurality of 
pages and displaying this information on said 
display screen. 

15. An audio/video network control system in which two 
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16. 



or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is term d a controller, 
and the units that are controlled are termed 
devices, 

said devices having display parts having 
manipulation information for changing the dis- 
play of the operation screen through operation 
by the user, and means for holding a menu con- 
sisting of a plurality of said display parts show- 
ing said operation screen, 
said controller having means for reading said 
display parts from said devices, and 
said menu of said devices being displayed on 
the display screen, and the display of said dis- 
play screen being changed through said opera- 
tion by the user according to said manipulation 
information. 

The network control system according to Claim 15, 
wherein the manipulation information comprises 
identifiers showing the other display parts. 



17. The network control system according to Claim 15, 
wherein the menu has configuration set parts, and 
the manipulation information comprises identifiers 
showing the other display parts. 

18. The network control system according to Claim 15, 
wherein the devices have a plurality of menus, and 
said menus are equipped with display parts having 
as manipulation information identifiers that show 
the other menus. 

19. The network control system according to Claim 15, 
wherein the devices have help menus that explain 
the functions exhibited by the various display parts, 
and said display parts have as manipulation infor- 
mation identifiers that show said help menus. 

20. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 

said devices having: 

means for holding display parts having manip- 
ulation information for changing the display of 
the operation screen through operation by the 
user; and 

menus consisting of a plurality of said display 
parts showing an operation screen for said 
devices. 



21. A controller in an audio/video network control sys- 
tem in which two or more units that handl one or 
more sets of data pertaining to vid o, audio, or 
information are connected via a transmission line, 
of these units, th one having a us r interface is 
termed a controller, and the units that are controlled 
are termed devices, 

wherein said controller reads from the devices a 
display part having manipulation information for 
changing the display of the operation screen 
through operation by the user, and a menu consist- 
ing of a plurality of said display parts showing an 
operation screen for the devices, 

said menus of said devices are displayed on a 
display screen, and 

the display of said display screen is changed 
through said operation by the user according to 
said manipulation information. 

22. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 

said devices having means for holding display 
parts that hold channel information indicating 
the channel of said transmission line and that 
constitute an operation screen for said devices, 
said controller having means for reading said 
display parts from said devices, and 
the channel of said transmission line being set 
according to said channel information in the 
handling of the data shown by said display 
parts. 



40 23. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or intonation are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 

45 and the units that are controlled are termed 
devices, 

said devices having means for holding a plural- 
ity of display parts that constitute an operation 

so screen for said devices, and configuration set 

parts that have channel information indicating 
the channel of said transmission line and that 
show a set consisting of a number of display 
parts from among said display parts, 

55 said controller having means for reading said 

configuration set parts and said display parts 
from said devices, and 

the channel of said transmission line being set 
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according to said channel information in the 
handling of the data shown by said display 
parts belonging to said configuration set parts. 

24. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 

said devices having means for holding a plural- 
ity of display parts that show an operation 
screen for said devices, and menus listing 
channel information indicating the channel of 
said transmission line, 

said controller having means for reading said 
display parts and said menus from said 
devices, and 

the channel of said transmission line being set 
according to said channel information in the 
handling of the data shown by said display 
parts belonging to said menus. 

25. The network control system according to Claim 22, 
23, or 24, wherein the display parts, configuration 
set parts, or menus have flags showing the type of 
stream. 

26. A device in an audio/video network control system 
in which two or more units that handle one or more 
sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 
controller, and the units that are controlled are 
termed devices, 

said devices constituting an operation screen 
for said devices and having means for holding 
display parts that contain channel information 
indicating the channel of said transmission line. 

27. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 
more sets of data pertaining to video, audio, or 
information are connected via a transmission line, 
of these units, the one having a user interface is 
termed a controller, and the units that are controlled 
are termed devices, 

said controller having means for reading dis- 
play parts that constitute an operation screen 
for said devices and that have channel informa- 
tion indicating the channel of said transmission 
line, and 

the channel of said transmission line being set 
according to said channel information in the 



handling of the data shown by said display 
parts. 

28. A device in an audio/video n twork control system 
5 in which two or more units that handle one or more 

sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 
controller, and the units that are controlled are 
10 termed devices, 

said devices having: 

means for holding a plurality of display parts 
that constitute an operation screen for said 

15 devices; and 

configuration set parts that show a set consist- 
ing of a number of display parts from among 
said display parts and that have channel infor- 
mation indicating the channel of said transmis- 

20 sion line. 

29. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 
more sets of data pertaining to video, audio, or 

25 information are connected via a transmission line, 
of these units, the one having a user interface is 
termed a controller, and the units that are controlled 
are termed devices, 

30 said controller having means for reading a plu- 

rality of display parts that constitute an opera- 
tion screen for said devices, and configuration 
set parts that show a set consisting of a 
number of display parts from among said dis- 

35 play parts and that have channel information 

indicating the channel of said transmission line, 
and 

the channel of said transmission line being set 
according to said channel information in the 
40 handling of the data shown by said display 

parts belonging to said configuration set parts. 

30. A device in an audio/video network control system 
in which two or more units that handle one or more 

45 sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 
controller, and the units that are controlled are 
termed devices, 

50 

having means for holding a plurality of display 
parts that constitute an operation screen for 
said devices, and menus listing channel infor- 
mation indicating the channel of said transmis- 
55 sion line. 

31. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 
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more sets of data pertaining to video, audio, or 
information are connected via a transmission lin , 
of these units, the one having a user interfac is 
termed a controller, and the units that are controlled 
are termed devices, 5 



more s ts of data pertaining to video, audio, or 
information are connected via a transmission line, 
of thes units, th one having a us r interfac is 
termed a controller, and the units that ar controlled 
are termed devices, 



said controller having means for reading a plu- 
rality of display parts showing an operation 
screen for said devices, and menus listing 
channel information indicating the channel of 
said transmission line, and 
the channel of said transmission line being set 
according to said channel information in the 
handling of the data shown by said display 
parts belonging to said menus. 

32. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 

said devices having content display parts that 
constitute an operation screen for said devices, 
show the contents handled by said devices, 
and list control codes specifying said contents, 
said controller having means for reading said 
display parts from said devices, and 
said controller using said control codes to 
direct the transmission and receipt of said con- 
tents to said devices in the operation of said 
content display parts by the user on said dis- 
play screen. 

33. The network control system according to Claim 32, 
wherein the content display parts have channel 
information indicating the channel of the transmis- 
sion line over which the contents shown by said 
content display parts are transmitted and received. 

34. A device in an audio/video network control system 
in which two or more units that handle one or more 
sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 
controller, and the units that are controlled are 
termed devices, 

said devices having means for holding content 
display parts that constitute an operation 
screen for said devices, show the contents 
handled by said devices, and list control codes 
that specify said contents. 

35. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 



said controller having means for reading from 
the devices the content display parts that con- 
stitute an operation screen for said devices, 
w show the contents handled by said devices, 

and list control codes that specify said con- 
tents, and 

said controller using said control codes to 
direct the transmission and receipt of said con- 
15 tents to said devices in the operation of said 

content display parts by the user on said dis- 
play screen. 

36. An audio/video network control system in which two 
20 or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
25 devices, 



said devices having means for holding display 
parts that constitute an operation screen for 
said devices, show the contents handled by 
said devices, and list control codes specifying 
said contents, and content configuration set 
parts that show a set consisting of a plurality of 
said content display parts, 
said controller having means for reading said 
display parts and said content configuration set 
parts from said devices, 
said content display parts being displayed on a 
display screen according to the information 
shown by said configuration set parts, and 
said controller using said control codes to 
direct the transmission and receipt of said con- 
tents to said devices in the operation of said 
display parts by the user on said display 
screen. 
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37. The network control system according to Claim 36, 
wherein the content configuration set parts have 
channel information indicating the channel of the 
transmission line over which the contents shown by 

so the display parts belonging to said content configu- 
ration set parts are transmitted and received. 

38. A device in an audio/video network control system 
in which two or more units that handle one or more 

55 sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interfac is termed a 
controller, and the units that are controlled are 
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termed devices, 

said devices having means for holding display 
parts that constitute an op ration screen for 
said devices, show the contents handled by 
said devices, and list control codes that specify 
said contents, and content configuration set 
parts that show a set consisting of a plurality of 
said content display parts. 

39. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 
more sets of data pertaining to video, audio, or 
information are connected via a transmission line, 
of these units, the one having a user interface is 
termed a controller, and the units that are controlled 
are termed devices, 

said controller having means for reading from 
the devices the display parts that constitute an 
operation screen for said devices, show the 
contents handled by said devices, and list con- 
trol codes that specify said contents, and con- 
tent configuration set parts that show a set 
consisting of a plurality of said content display 
parts, 

said content display parts being displayed on a 
display screen according to the information 
shown by said configuration set parts, and 
said controller using said control codes to 
direct the transmission and receipt of said con- 
tents to said devices in the operation of said 
display parts by the user on said display 
screen. 

40. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 

said devices having means for holding display 
parts that constitute an operation screen for 
said devices, show the contents handled by 
said devices, and list control codes specifying 
said contents, and content menus that show 
the functions of said devices and codes that 
indicate one or a plurality of said display parts, 
said controller having means for reading said 
content menus and said display parts from said 
devices, 

said content menus being displayed on a dis- 
play screen, and 

said controller using said control codes to 
direct the transmission and receipt of said con- 
tents to said devices in the operation of said 



10 



15 



20 



25 



30 



35 



40 



45 



50 



55 



display parts by the user on said display 
screen. 

41. Th network control system according to Claim 40, 
wherein the content menus have channel informa- 
tion indicating the channel of the transmission line 
over which the contents shown by the display parts 
included in said content menus are transmitted and 
received. 

42. A device in an audio/video network control system 
in which two or more units that handle one or more 
sets of data pertaining to video, audio, or informa- 
tion are connected via a transmission line, of these 
units, the one having a user interface is termed a 
controller, and the units that are controlled are 
termed devices, 

said devices having means for holding display 
parts that constitute an operation screen for 
said devices, show the contents handled by 
said devices, and list control codes that specify 
said contents, and content menus that show 
the functions of said devices and codes that 
indicate one or a plurality of said display parts. 

43. A controller in an audio/video network control sys- 
tem in which two or more units that handle one or 
more sets of data pertaining to video, audio, or 
information are connected via a transmission line, 
of these units, the one having a user interface is 
termed a controller, and the units that are controlled 
are termed devices, 

said controller having means for reading from 
the devices the display parts that constitute an 
operation screen for said devices, show the 
contents handled by said devices, and list con- 
trol codes that specify said contents, and con- 
tent menus that show the functions of said 
devices and codes that indicate one or a plural- 
ity of said display parts, 
said content display parts being displayed on a 
display screen, and 

said controller using said control codes to 
direct the transmission and receipt of said con- 
tents to said devices in the operation of said 
display parts by the user on said display 
screen. 

44. An audio/video network control system in which two 
or more units that handle one or more sets of data 
pertaining to video, audio, or information are con- 
nected via a transmission line, of these units, the 
one having a user interface is termed a controller, 
and the units that are controlled are termed 
devices, 
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said devic s having means for holding display 
parts that constitute an operation screen for 
said devices, and cont nt menus that show th 
functions of said devices and codes that indi- 
cate one or a plurality of said display parts, 5 
said controller having means for requesting the 
right to use said devices, and 
said devices transmitting a menu list containing 
identifiers for the display parts that constitute 
said menus of said devices to said controller 10 
when said devices grant said usage right. 

45. The network control system according to Claim 44, 
wherein, when there is a change in the display 
parts within the devices, said devices transmit said 15 
changed display parts to the controller. 

46. The network control system according to Claim 44, 
wherein, when the controller relinquishes the right 

to use the devices, said controller notifies said 20 
devices that the usage right is being relinquished, 
and said devices relinquish the channel of the in- 
use transmission line at the directive of said control- 
ler. 
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Fig.4 
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